Sunday, April 22, 2018

Crackpot Index

This index was originally meant for science articles some would toss at me in the hope that I'd help them get their ideas published. Most such submissions were a waste of my time, but occasional students showed potential. Ignorance is curable. Ego might be too, but I'm not interested.

This index isn't original material. It qualifies as derivate material, but I've lost track of where I got the first idea. It was many, many years ago during the early days of the internet, so anyone who help me reference this properly would be greatly appreciated.

Nowadays I'm tempted to write a political version of this for some of the stuff I read. No doubt many others have written them, but there might be some value is cataloging them all. It might reveal how we gist each other's ideas. 8)

Simple method for rating potentially revolutionary contributions to science.

Basic Points: (Start with -5 points as a starting credit.)
  1. 1 point for every claim that is widely agreed to be false.
  2. 1 point for a work that relies upon anecdotal evidence.
  3. 1 point for capitalization of concept words not at the start of a sentence or 5 points if concept words are all caps.
  4. 2 points for every statement that is clearly vacuous or logically inconsistent.
  5. 3 points for an author working in isolation.
  6. 3 points for scientific effect that are always at the very limit of detection.
  7. 4 points for using a thought experiment that contradicts the results of a widely accepted real experiment.
  8. 4 point for the first publication being pitched directly to the media. 1 more point if they use a personal blog.
  9. 5 points for work that relies on a multi-century belief claimed as credible because it has endured.
  10. 5 points for each mention of a famous scientist as an authority or -1 point for proper and relevant citations.
Bonus Points:
  1. 10 points for each claim that a well established theory is “fundamentally misguided” or “only a theory.”
  2. 10 points for pointing out that the author has gone to school, as if this were evidence of sanity or authority.
  3. 10 points for a theory description explaining how long the author has been working on it.
  4. 10 points for sending work to unknown people requesting they not reveal details for fear the ideas will be stolen.
  5. 10 points for offering prize money to anyone who proves and/or finds any flaws in the theory.
  6. 10 points for each statement along the lines of "I'm not good at math, but my theory is conceptually right, so all I need is for someone to express it in terms of equations".
  7. 10 points for arguing that while a current well-established theory predicts phenomena correctly, it doesn't explain "why" they occur.
  8. 10 points for each favorable comparison between the author and a famous scientist... written by the author.
  9. 10 points for claiming that the work is on the cutting edge of a "paradigm shift".
  10. 20 points for suggesting that the work is deserving of a Nobel prize.
  11. 20 points for every use of science fiction works or myths as if they were fact.
  12. 20 points for defensive language describing real/imagined ridicule aimed at author's past work.
  13. 25 points for each use of the phrases "hidebound reactionary" or "self-appointed defender of the orthodoxy".
  14. 30 points for suggesting that a famous scientist secretly disbelieved a theory they otherwise publicly supported.
  15. 30 points for suggesting a famous scientist, in their later years, was groping their way towards the ideas the author now advocates.
  16. 30 points for claiming that a theory was developed by an extraterrestrial civilization.
  17. 35 points for allusions to a delay in the work while the author spent time in an asylum, or references to the psychiatrist who tried to dissuade the author.
  18. 40 points for claiming that the "scientific establishment" is engaged in a "conspiracy" to prevent the work from gaining its well-deserved fame or respect.
  19. 40 points for comparing those who oppose the work to a modern-day Inquisition or to Nazis.
  20. 50 points for claiming that when the theory is finally appreciated, present-day science will be seen for the sham it truly is. (30 more points for fantasizing about show trials in which scientists who mocked the theory will be forced to recant.)

Friday, February 28, 2014

Human Markets

If one starts from the assumption that the resource optimization problem is unsolvable in theory for large groups, but partially predictable for small groups, one can cast some of our social institutions in a light that demonstrates their roles as these partial solution engines. The most obvious one to any westerner is the classic marketplace/bazaar where people barter.  For this pig I want...

There are other kinds of markets, though. This is most easily demonstrated if we stick to the most abstract definition of a market where it is described as a system by which people engage in exchange. The trade of goods and services directly as in barter or indirectly through the use of money is just one style that works for alienable stuff. There are also two styles of gifting that fit the definition with the first being something like charity with no expectation of return and the second being prestation where a return of equal or greater value is very much expected. Everything from commodities to experiences can be exchanged in these ways and we occasionally mix them with varying results. All these methods, though, tend to support the kind of market we might call Commerce. Exactly which rules apply depends on the stuff being traded. For example, I might buy and sell what I produce from a farm in a cash-based marketplace, but if there is a famine, I might be expected to gift what I produce as charity or with a delayed expectation of reciprocity. In another example, I might never be permitted to trade for kidneys using cash, but in some places I might get away with brokering such deals through barter. Getting the trade rules right can be tricky when crossing cultural borders and those who get them wrong can easily wind up being chased by a lynch mob.

There are other kinds of exchange, though, and these also point to partial solutions to a 'resource' optimization problem. Consider our concept of Justice. If someone does wrong to my kin I can respond in kind. This is a classic form of justice practiced by humanity over countless generations, but it motivates a negative sum game when done in the presence of markets for goods and services. If one market participant steals from my brother and I steal from them in return, all participants will consider the marketplace to be risky and spend resources guarding their resources. If this continues too far, the marketplace collapses when participants find it cheaper to stay away or trade elsewhere. The solution to this, therefore, is to trade with the rules of justice themselves. If one participant can dominate a marketplace and dictate the rules, the 'trade' is simplified to a simple decision of whether to participate or not. If no one dominates, then the 'trade' is in the rules and expectations regarding responses. Is it an eye for an eye and a tooth for a tooth, or is it a hand for stealing a loaf of bread and a life for poaching on the King's land? Are infractions decided vigilante style or are judges, juries and executioners required? Rules of justice that enable the other markets have the potential to make participants rich, well defended, and well off, so this is important stuff to decide. Whatever the rules are, this market is called Justice.

Next, consider our inclination to seek consensus when making decisions at a group level that imply impacts to all individuals in the group. Whether these groups are families, communities, or nations is just a matter of scale that can impact the practicality of how we determine a consensus. Whether these groups decide through votes, debates, gift giving, or shouting matches is just a matter of process and can impact the perceived validity of the resulting consensus. Since these decisions could impact the rules of Justice and what is brought to market for Commerce, there is a strong motivation to trade support for decisions. In the western sense we might call this market Democracy, but it need not look much like the Greek ideal. In the most abstract form, some people get to have a say in determining the consensus and they trade their willingness to support one decision for another. In this market, one Senator might support the spending of money on a bridge to nowhere if another Senator supports funding for a sports arena. They might also trade a change to immigration law for tougher penalties for certain crimes. Whether these trades are moral or not is judged by society, but the trades happen, thus a market exists and determining the rules governing these trades is just another form of trading.

So far, we have three basic markets and they are enough to cover most of human history. There are no clear lines separating them, so it is best not to think of them as boxes into which a particular exchange must fit. There are far too many counter examples for this to be even possibly true. Just think of a general case of a legislator accepting bribe money (Commerce) in exchange for a vote (Democracy) to make a particular kind of trade illegal (Justice). It doesn't take much to search recent history in any country to find examples.
  • Commerce
  • Justice
  • Democracy
There is a fourth market that is worth finishing up with here. It is very young and proving to have a dramatic impact on how the world works because it solves a particularly difficult part of the general resource optimization problem. This is the part where we assume planners know how things work well enough to predict their futures. There are huge issues with this assumption even in the modern world because we simply don't know how to predict the future with any precision. There are certain problems, though, were we've come to be pretty good at such predictions and this improvement isn't a matter magical thinking. An odd thing happened in certain parts of northern Europe a few centuries ago when people began to trade what they knew from experience using a set of rules that later came to be known as the scientific method. Descartes referred to them as 'The Method' and they are born from a field known as natural philosophy. The net result of this trade has been the growth of attempts to know what is true that are gifted to all, but there is a strong expectation of reciprocity with a very odd twist. Instead of simply expecting more or greater gifts in return, gift givers expect their gifts to be thrashed, mashed, and tossed in the trash if they don't live up to the agreed upon expectation that they must help explain and predict the world around us. This market is the one called Science, but its participants include everyone and most innovations start with the engineers.

If we define modernity from the birth of this fourth marketplace, we can point to a time on the calendar that occurred in the mid to late 17th century in northern Europe and later decades elsewhere. The impact of this fourth market is hard to understate when one notices that the Industrial Revolution kicked off a few decades later after people had some time to adopt and adapt to the new market. Look at the history of the early adopting nations and you'll see changes to their wealth and status in Europe even before the industrial age.
  • Commerce
  • Justice
  • Democracy
  • Science
On the general problem of solving resource optimization problems, therefore, it seems humans do this best by inventing markets that partially solve certain parts of the generally unsolvable problem. Democracy in its most abstract form has been with us since before we were human and Justice followed in short order no doubt as our numbers and mental capabilities grew. Commerce, however, is probably necessary for something like the agricultural revolution that occurred short after the ice melted. Connecting these two, though, is a stretch not supported by evidence. There is good reason to believe humans have been trading goods for almost as long as we've been human and some point to this as a useful demarcation between us and the other Great Apes. If so, the first three markets are truly ancient with only Commerce being the one that distinguishes humanity from the other animals.

Is it a lack of humility to argue that humans have invented a fourth market after countless generations of relying on our traditional triplet? I think not. The fact that our population has exploded from roughly half a billion to over seven and still climbing proves something dramatic happened. A future archeologist from an alien race might have difficulty finding anything in the rock strata to explain this explosion beyond the obvious fact that it happened, yet those of us here living it know the cause. We figured out how to predict many things we couldn't predict in the past. We figured out how to put men on the Moon and we figured out how to eradicate Small Pox from the face of the Earth. We figured this out because we invented a new system and it has proven to be wildly successful by being incredibly useful. This system is just another market, but humanity doesn't invent those very often. That last time we did... we became human. What will we become now?

Wednesday, December 18, 2013

Surveillance Limitation as a Necessity for Freedom

I've been watching the social debate over the last few weeks regarding the NSA and their scooping up of phone conversation data. This is about the details of what the NSA does that Edward Snowden revealed when he released classified information. I have a number of friends who obviously think Snowden did the right thing and I've said once or twice that I strongly disagree. I've given up trying to convince them for now because they appear to be reacting mostly from fear and there is no way to win them over while that is true. However, I'm going to record thoughts here for later use when they calm down a bit.

  1. The NSA program is much like a CIA program that got shot down a few years ago when it was revealed too, thus this is a whack-a-mole game. IF NSA surveillance of this type is made illegal, I strongly suspect it will go underground again and appear somewhere else run by people who have learned lessons from this experience. Players in this game LEARN and that is especially true of the mole if it doesn't want to get hit. If we continue this way, we will unintentionally train the mole to avoid us while it continues to thrive. This reminds me of a lesson my mother once taught me regarding some behavior of mine she didn't like. She was very careful to use 'Don't let me catch  you doing that.' only in situations where she meant precisely that. Don't let me catch you.
  2. If my freedom relies upon an elite group not knowing details about me, I'm in serious trouble. I won't know if they do or not unless I have access to everything of theirs and KNOW that I do. How could I know what an elite group knows about me otherwise? Making it illegal for them to collect the information doesn't stop them from breaking the law, so I would have to be very trusting not to verify. The problem is that with that power to observe them, I become a member of an elite with awesome powers. I become the very danger I'm worried they might become. What's to limit me, then? I can't give this awesome power to someone else without creating an elite group that knows too much, thus I have to protect my freedom myself, right? Everyone does. Follow the logic on that and we wind up with a requirement for sousveillance and an abandonment of the notion that my freedom relies upon an elite group not knowing much about me. What my freedom REALLY relies upon is ALL of us being able to know what the elites know. We don't have to limit what they know. We have to improve what WE know.
  3. Does it really matter what an elite group knows about me if most everyone else can know it too? Isn't it more important that I focus upon what they can DO with that information? I'm thinking of eating fish for dinner tomorrow night. Now others know. So what?! It doesn't matter until someone tries to do something to change my thoughts or plan.
I found an old image from the Santa Barbara Zoo in one of my albums. I like the black swans as a reminder that sometimes people are simply wrong about something even though they all mostly agree on the correctness of their opinion. Sometimes a surprise happens and one encounters a shocking counter-example. Whether people will be shocked by the dissonance between their desire to protect their freedom and their plan to blind elite groups remains to be seen, though. I'm doubtful. People can suffer dissonance for a long time until forced to confront it and I don't know a way to do it. Besides, forcing people to face this is mildly distasteful to me.

Anonymity has to die as the dodo did.

We simply need to know more if we are to protect our freedom from those we've tasked with protecting us from bad people. Those elites, often as not, serve as our protectors. We'd be fools to blind them.

Thursday, December 5, 2013

What Humans Do Best - Part 4

I should summarize things a little at this point so I can show the point toward which I'm driving with the earlier sections.
  1. Initially I focused upon an ideal optimization problem for resource allocation in family groups because there is a class of problems people resist having automated and I run into this resistance at work and other social settings. In this class is the problem of assigning resources that could impact our children's health, education, and other necessities. In a theoretical sense this problem is solvable with linear algebra, but for technical reasons it is only partially solvable. The issue is the family planner doesn't have the information they need to do a precise job, but they know that isn't strictly necessary and solve a simplified problem and that is usually good enough. 
  2. Next I focused upon the simplified problem in more detail using an example and showed how we further reduce it in practice. This reduction is necessary because the simplified problem isn't really solvable and the people who depend on the family planner know it and resent solutions that do not favor them. In that resentment the planner loses even more information. It also hints at a couple of mathematical issues that go beyond the limits of knowledge too. 
  3. The last part focuses upon the mathematics issues to show even the simplified, reduced problem involves delegation and breaking it up into smaller problems for each planner. It then finishes by turning the problem around to show that a planner winds up serving an elite group and really can't operate any other way. As the problem is reduced and subdivided each elite becomes smaller and better served, but at the cost of control and efficiency.
In this section I want to knit these back together by pointing out that our natural inclination is to subdivide resource allocation problems among many planners, thus we really should be looking at what it means to solve N problems (N planners are involved) each juggling Pi resources where i runs through the list of planners. In the limit we might assume N to be the number of individuals in a community, but it could easily be smaller than that if people organize around nuclear or extended families. In that case N would be the number of 'atomic' units that plan and economize resource usage. If the average 'family' size was four people, N would be 25% of the community's population size. In this sense, the atomic planning groups are probably the best working definition of what a 'family' is when it comes to human communities.

Turning the problem around is best done in a step-wise fashion. Let's start by assuming we live in a community with one planner who actually means well by all of us and seriously tries to serve the largest elite they can. There is a way to do this that mollies the people in the corners of the preference space often enough to keep the peace and some leaders employ the strategy. The trick to it is to take a page from game theory and use a bit of randomness in the solution offered over time. If done well, the other players in the game will find it difficult to choose between the strategies they might play in response to the planner. That makes the non-planning players somewhat predictable even if one doesn't know their preferences in detail. To get the most people involved, the planner will target a particular solution and then include a span for each of the variables in the domain. The resulting range of solutions will probably map close to the target solution, but only if we can make the usual assumptions about continuity and derivative smoothness. If we can do that, the solution range might be an n-ball around the target solution on first approximation where the ball is made as large as planner can manage in the practical sense.

Now consider what happens if the planner fails to mollify a group of people and they rebel successfully and choose their own planner. The first planner need no longer include them and might shift their targeted range to another region of the solution space. The second planner will target something closer to the elite who chose them. The result is two n-balls with a moderate possibility of overlap since each planner might still want to steal people back to their group and might try for those who were barely motivated to rebel. The two groups will otherwise operate as separate communities each optimizing resources as best they can, but without the power to force the other to make certain choices. More people will feel included in an elite group and more people will be covered by a randomizing strategy that keeps them somewhat mollified. The cost, however, is there are now two solutions that might not be reconcilable and the two communities might fight over scarce resources. History is full of examples where this has happened, but it is also full of alternate examples where the communities found an alternate path that avoided the fight. It is the alternate path we should consider in detail, but it too is a form of optimization.

If two planning communities are roughly equally matched in a potential fight, it doesn't take a genius planner to realize that a fight will destroy more resources than it might gain. In other words, there is a cost to violence. The alternatives to this cost, therefore, are obvious. One can simply do nothing and avoid the fight; one could try to steal resources and get them to defensible positions; or one could offer to trade voluntarily and possibly avoid the anger that comes with the other two alternates. Doing nothing isn't uncommon. We call it self-sufficiency most of the time. Theft isn't uncommon either, but it leads to a negative-sum game where planners must devote resources to defenses and that makes it unsustainable over the long haul. Violence usually follows when defense costs get high enough to justify war costs as a replacement.

The winner, though, is voluntary trade as that can lead to a positive sum game and a form of optimization where control is less than complete. If two groups can trade resources they get at least some of what they want, but will have difficulty knowing in advance how much they will have to give up to get it. Instead of not knowing what resources to assign and to whom, they will also not know how much they will have to trade away to get the ones they think they are missing in their plan. What they will know, though, is that they can acquire a resource if they are willing to pay enough and that means they can set up their own randomizing strategy between self-sufficiency and trade in a particular resource. In that way, they can moderate the price they pay by putting a ceiling on it at the cost of some other resource they might develop instead. Their optimization problem turns from finding point solutions to the finding of solution space regions, but they were already doing that because each planner was motivated to maintain the power base they needed to keep their job. In this sense, the simplification and reduction of the original optimization problem leads naturally to the trade solution when atomic planning groups break up. Markets should be expected in human societies involving multiple planners.

Finally, lets consider what happens if communities break up into more than two groups. When N groups all vie for resources, the costs of fighting are even more difficult to bear as they are often a larger fraction of the available resources for a small group than for a larger group. In the limit of individuals, they simply don't have the wealth necessary to fight for all their preferences when facing N-1 other planners unless the starting conditions are seriously out of balance. If they are that far out, it would make some sense for the N-1 others to steal as rapidly as they can to force something closer to a balance or force the single player to devote their wealth to defense and wait them out much like one would when employing siege tactics. Absent large imbalances, though, small planning groups are best served by markets if the markets function transparently enough to make their optimization problems moderately predictable. A market that isn't transparent isn't motivating as it doesn't solve their coordination problem with other groups leaving them with the original options of doing nothing, stealing, or violence. Transparency in a voluntary market is the minimum necessary requirement.

I've been purposely ambiguous when I use the term market because humanity has more than one kind of market it employs for trade and each uses a different coordination technique and exchange medium. Some of the market types are truly ancient and barely recognizable to those who think only in terms of money and barter. Others are so new they too are barely recognizable, yet they are changing the world so rapidly with new voluntary trade that we are left dizzy after our heads are spun around. I'll finish up next time with my views on each of the markets I can see and leave open the question of whether there are more.

Sunday, December 1, 2013

What humans do best - part 3

When hunting for solutions to problems with many variables it is important to examine two aspects of the process we use. The first is the search technique. How do we determine where to start looking and how do we adapt to the results we find to try again? Our search technique is recursive as one would expect of humans who learn from their errors. The second is our determination of what is optimal. How do we know that nearby alternative solutions aren't better? This translates as knowing when to stop searching, thus it is a recursion termination issue and falls into the problem of knowing when a computation is complete.

Consider the search technique first, though. If a problem to be solved has only one variable it is easy to describe an algorithm that works at finding a solution fairly well. Assume one resource is used and only a finite amount of it is available. If we use none of it we assign the variable a zero in the function that determines what we do with it. If we use all of it we assign unity to the variable. Some other value implies fractional use of the resource. The nature of the function is purposely left ambiguous here because it depends on the utility one derives from using the resource. It is whatever it is and doesn't matter much. What matters here is the domain of the function and not its range.

A typical approach one may use if one can make a few assumptions about the non-pathological range of the function is a binary search method. One might start in the middle of the domain and calculate the function along with the ¼ and ¾ values. If the values to either side of the middle produce better answers than the middle, we recalculate the next iteration using the better value as the middle and add and subtract 1/8 from it to get the extremes and then examine the results again. If we know more about the range of the function and can assume the existence of one or more derivatives at the middle point being calculated this is similar to calculating the two extremes, but we might be able to delay the calculations for awhile. Either way, though, the information in this technique creates a shifting net of calculations that examines the output of the function through iterations and narrows in on a solution using our belief that the marginal rates of substitution must be equal. In this case, there is only one commodity, so what is being traded is our inclination to prefer one solution over another.

Now consider problem termination. In one variable we are already done. The iterations end when we don't know where to go next in our screening of the domain. When the differences between the middle value and the two extremes drop below some threshold, we are done and call the middle value the solution. There is no guarantee the solution is globally optimal, of course, but it is probably a good candidate. We might try a few more times using random starting points in the domain instead of the middle, but if we choose a finite number of retries, that problem also ends the same way.

Things become more complicated when the number of variables grows, though. Even at two variables we have an issue with the search technique. We use the same fractional domains [0, 1] for each of the variables and define the 'middle' as (½, ½). What are the extremes to be calculated in the approximation of a derivative, though? In two dimension a tangent line slope isn't enough to know the nature of the function range as it was for one dimension. We need something equivalent to a gradient operator where we don't have to make too many assumptions about continuity and the existence of derivatives along an arbitrary curve through the domain. We get this by creating a mesh of points to examine with the middle point being the start point for each iteration. The mesh points are always defined relative to this middle and get closer to it as the iteration step increases. In two dimensions we might pick the corners of a square and the midpoints on the edges of the square and use the middle of the square as the foot fo the iteration. That means a jump from one dimension to two increases the number of calculations from three to nine each time. It's not hard to see where this goes if the number of dimensions becomes much larger. In the computational sense, the problem is intractable and we won't bother trying to solve it that way.

In two dimensions we might use our nine point mesh and be able to terminate our search much like we did in one dimension. We might even use a similar approach to test whether the solution we find is locally or globally optimal. In a theoretical sense, the search techniques and termination strategies can be the same even though we know the problem will take a lot longer to calculate if the function is messy.

In higher dimensional problems a more likely search technique we will use involves choosing random points near the foot of the iteration and examining them as our mesh. We can choose a finite number to test and approximate the gradient operator. Doing it this way keeps the problem tractable, but sacrifices exactness regarding where the next iteration of the search can go. If through randomness we wind up testing one region of the domain better than another, our next iteration will be biased. We mitigate this risk through statistical measures and only move to the next step when the quality of the test points is sufficient. In this, though, we make it a little more difficult to know when our problem ends. We can be reasonably certain it WILL end, but it is hard to predict when beyond a probabilistic statement.

It is in the statistical measures that there are issues, though. We should weight points near the foot stronger when determining the value of the gradient operator near that foot. This is similar to what we do when calculating the slope of a tangent line. The close the points are to each other in the domain, the more likely the slope of the line connecting them IS the slope of the tangent line at the foot. We are merely extending the concept and the continuity and derivative assumptions if they are available to higher dimensions. What does it mean to be near a foot in a higher dimensional domain, though? That requires something similar to a distance formula which implies the existence of n-balls in the domain that map to something reasonable in the range. The problem is that there are 'corners' in our problem if there are two or more variables. The corners are underweighted in the test and might contain legitimate solutions we gloss over. In fact, the corners are a serious problem. It is easy to show that the ratio of the volumes of an n-ball to the n-cube that contains it drops to zero as the number of dimensions increases in the limit. An n-cube is mostly corners relative to the foot in the middle. Try it with squares, cubes, and hypercube formulas and things don't look too bad, but when the number of dimensions is higher than five the point is quickly demonstrated.

Can this underweighting of corners be mitigated? Sure. We can increase the number of test points with the number of dimensions and adjust the weighting function. Unfortunately, though, we restore the intractability of our problem at some point. Exactly where depends on how many variables are required and how much we wish to avoid over-reliance on a technique that underweights corners. THAT means we are also back to solving the less than ideal problem and choosing instead to settle for an approximate solution where we can't know if it is globally optimal even in a probabilistic sense.

If you are the head of a household and the solution you seek is the typical family economic planning associated with having enough to eat for everyone, there are a limited number of variables that matter. In the simplest sense one must provide enough carbohydrates and proteins to everyone. We can hope the rest works out regarding minerals and vitamins and all that or try to find local solutions on occasional to make up for deficiencies. Exactly how we approach this doesn't matter here, though. What matters is that we CAN approach it and that we typically do so by diving the effort among the family members. One might focus on gathering carbohydrates and optimizing what THEY do in order to accomplish that. Another might focus on hunting protein and optimize for that. The combined solution makes use of parallel processing among the minds of the family and makes difficult tasks a little more tractable, but it occurs through the sacrifice of any knowledge about the global optimum solution. Fortunately, we usually don't care. A local solution that keeps the family alive is good enough most of the time. Global optimums only matter under times of great stress.

If you are the central planner of a tribe or community, though, the problem rapidly balloons to many, many dimensions. The more your community gets along with each other, the simpler the problems gets, but only because THEY coordinate their preferences for you. This is the core concept behind group identification I suspect. A nation is such a group of people who CHOOSE to coordinate preferences in order to facilitate the finding of solutions to resource allocation problems. If they don't make that choice, though, or only make it in the weakest sense meaning they won't steal from each other when they don't like a particular solution, the ideal problem isn't solvable in any sense of the word. Even the reduced problem intractable. Any central planner who makes the attempt is most likely going to find solutions that are relatively good for a subset of the community with preferences near the solution. The members in the corners of the preference space, though, will not be satisfied and might actually be quite upset. Since corner dominate in large dimensions this could mean that most members of a community of any size dislike a planned solution and would view it as serving the desires of a elite subset.

This essay is about what humans do best, though. Apparently what we do best is serve an elite subset of our communities with a focus upon our families. This makes biological AND cultural sense since we benefit in the evolutionary sense when our off-spring prosper relative to other people. Remember that our primary competitors on Earth are other people and this helps make sense of why people will resist surrendering this problem to automation. Yet, in a sense, we DO surrender it. We do so through task delegation to make the problem moderately tractable and when we voluntarily trade with other people outside our groups for which we plan. We can't optimize the resources used by those outside our groups very well because we can't dictate their usage, but there IS a mechanism by which we have managed to accomplish partial control. We trade what they value and lure others to optimize in the way we desire and that implies our markets play a critical role in finding solutions. More on that next time.

Thursday, March 7, 2013

What humans do best - part 2

There are a few issues associated with the focus on the ideal case with the main one being that it isn't the problem we actually try to solve. It is much more common that we attempt to solve a simpler sub-problem. Reality rarely gives us all the information we need for the ideal case, so while we might want to believe in the Platonic ideal for the problem, we work with mundane versions.

Earlier I asked what you might want for breakfast 10 years from now. Few people will bother to collect this information based on their belief that it doesn't really matter. There is little doubt you will want breakfast of some kind on that day, but the precise preferences you have don't have to be known in great precision. If you are a vegetarian, we might exclude sausage from the list of your preferences and from that exclude the means for getting sausage to you. If you are vegan, the list of exclusions is even more severe seemingly simplifying the problem. If you always have orange juice for breakfast, that is another kind of simplification. In both of these cases, we can set part of the solution to the overall problem without any further thought. For a more complicated example, consider what we would do if you tend to rotate between typical breakfasts like omelets, pancakes, and oatmeal. That would be enough to set probabilities at a minimum and that might be enough to get close to a solution.

Simplifying the economic problem this way isn't without consequences, though. The primary one is easy to explain. If I don't know what you want for breakfast 10 years from now, but do have enough information to make reasonable guesses, I might guess right or I might guess wrong. If my planned solution to the problem includes enough resources to provide for omelets, pancakes and oatmeal you will get what you want 10 years from now. Unfortunately, though, I've wasted resources in accomplishing that because you'll pick one of them and not the other two. Those are resources I might have put to better use serving preferences for someone else. What happens on the day when I fail to serve their preference and spent too many resources serving yours? How long would it be before someone thought that you had an unfair advantage in the plan? To avoid that, maybe I should just ensure enough is produced to cover all the probabilities so no one complains? That just ensures most of what gets produced ISN'T optimal. Perishables that don't get consumed get wasted and we might think we can live with that, but what about spending the resources used to produce that wastage on coming up with truly new products and services? What about innovation? Would there be any?

Yet this is pretty close to what we do when optimizing our resources for our families. We reduce the big problem to a smaller one and simply try to do our best. I don't know what you want for breakfast 10 years from now. Neither do you probably. Who cares? What do you want for breakfast tomorrow? If you want something exotic that I needed years to prepare to get, too bad. You aren't going to get it because I'm not omniscient. Next problem please.

Unfortunately, this limitation isn't enough to be convincing. It is vulnerable to the 'We are humans' counterargument because our computers aren't human. If we augment our data collection and processing capabilities, surely we can solve the harder version of these problems with more variables in them. Actually, we can't and the worst thing about it is that a solution could be right in front of us and we wouldn't know it in advance. The problem doesn't lie with knowing we have a solution. It lies with knowing we have an OPTIMAL solution. Remember that we are trying to optimize in the ideal case and that means we are trying to avoid waste. That is a big part of what we mean by economic efficiency, after all.

If you know the mathematics, you might be tempted to say it is just a matter of taking differences between neighboring solution attempts and using them to approximate the derivatives in a gradient. If you know a bit more, though, you'll remember that we must be careful about assumptions regarding continuity of functions. The ideal case solution is actually a function of many variables with time as a parameter of that function's curve. There is no reason to assume the function is continuous in all the variables, let alone continuous over long periods of time. In fact, it is probably a terrible assumption. How many pregnant women continue to want pickles and ice cream after giving birth? Our lives are full of discontinuities, so our preferences should be too.

It is worse than that, but I'll save the next material for part 3. We must face two inconvenient truths forced upon us by the mathematics of multi-variable problems.

Saturday, March 2, 2013

What humans do best - part 1

I've been thinking on something I heard today in a economics podcast where the speaker was talking about what is likely to happen to our jobs in the coming years. He made the case that most of them will suffer the same fate as farm jobs and for a similar reason. It isn't that they will all be sent overseas. It is that they will simply cease to exist because no human will do the work anymore. This wasn't news to me since I see it at work. In fact, it is my job to make it happen. The blunt truth is that I work to eliminate tasks from the lists worked by people by automating the work. The jobs I impact don't usually go away, but they certainly do change.

What caught my attention was the speaker's focus on the kinds of things we like to think that we like doing. What can humans do well that sets us apart from the tools we fashion? It used to be that no computer could beat the best human chess player. That is no longer true. It used to be that no computer could safely drive a car in a complex urban environment. There are very bright people demolishing that belief right now and doing a good job of it. Offer up a physical task and there are many who want to tackle it and automate it so the take away lesson is to be cautious of claiming any reserved turf at all.

There was a class of problems, though, that might be immune. It isn't that we can't automate solutions to these problems, it is just that we probably won't. These are the problems we LIKE to solve. They don't have to have much else in common except the simple fact that we like them, thus the hurdle to automating them is higher in the sense of the costs involved. Someone automating drudge work like I do might be met with concern from those suffering the changes we bring upon them, but after it is over and their job descriptions adapt to the new reality (assuming they still have the job), we usually meet with smiles and appreciation and a desire for us to go away so it doesn't happen again any time soon. However, if we try to take on a task they seriously enjoy, they will fight before and after and impose extra costs upon any who would pay us to do the deed.

A psychologist might examine why we like certain problems even to the point of resisting giving up the work when others can do it better, cheaper or faster. They might examine motivations and rewards. A biologist might examine the same behavior and study survival odds of those who keep the tasks compared to those who don't. Each of those might be interesting paths, but the one that caused my ears to perk up was the biological one as it relates to the problem of caring for one's family. All animals face this kind of problem and different solutions arise with different species, but among humans there is evidence that our oldest solutions were overlain by newer ones specific to certain settings and when that happened modern humans became what they are today. We became a rapidly growing species that displaced all other hominids and is in the process of displacing much more distant relatives to such an extent we should refer to modern times as a mass extinction.

In the ideal case, the problem one faces with the simplest task of feeding oneself and procreating in a paleolithic era with associated tools is still complicated, but it isn't much more complicated than other mammals have to solve. Our task was a touch more complicated because our primary competitors were other hominids and ourselves. When the others were gone, we faced a positive feedback loop by competing with each other.

Ideally, though, the problem of meeting the basic needs of a human family is all about providing food, water, shelter, and security. How we solve for all the preferences of our family members varies considerably depending on the means we have available and the relevant information we need. If we have it ALL, though, the problem resolves to one of optimization. If you are the head of a family it isn't hard to imagine that the depth of your knowledge and ability to think through all the options associated will all available resources is how you find the optimal solution. If your family members are also talented, you might delegate or share the planning tasks. How many physical brains are involved in the planning effort doesn't really matter. They key measure is how the preferences are communicated and resources assigned. If several people are involved and closely coordinating they function effectively as one person. If they are willing to take food from a baby that has more than it needs to give it to another baby that doesn't have enough, they are optimizing in this fashion.

From a biological perspective, it shouldn't surprise anyone why we like to solve this kind of problem ourselves and only surrender it grudgingly. We are the children of past generations who solved this problem well enough to procreate. We should have an evolutionary inclination toward wanting to solve these problems for the simple fact that those who don't want to slog through the hard work are less likely to have and raise kids.

From a biological perspective, it shouldn't surprise anyone why some of us get annoyed with parents who do NOT like to solve these problems. If you are the kind of person capable of caring about the welfare of children who are not your own, you might be tempted to expand your optimization effort to include them for moral reasons. Basically, the other parents are freeloading on your good will. While the problems are easy to describe they are hard to solve. Even if we find solutions that don't deprive our own families too much to cast our wider net of concern, it is still hard work we don't accept easily.

The optimization problem in a paleolithic setting is simpler than in a modern setting. There were vastly fewer people then and our knowledge of how things worked was much more limited. There is no doubt our ancestors knew things we have since lost, but they didn't need to know as much as we do and absent many of the resources we now have available it is easy to argue the knowledge wouldn't have done them any good anyway. What need did someone have for trying to accurately predict the position of Mars in the sky to within one arc-minute if they lived 20,000 generations ago? Their problems were still very complex, though, even if one could know all the details needed as inputs. Some did it well and we are their great-to-the-nth-grandchildren.

There are two twists to this type problem that make them even more difficult. The most obvious one is that the planners don't have ALL the information. F. A. Hayek pointed out that they can't. In fact, they can't EVER. There is the annoying little possibility that approximate information might not be good enough too because the optimal solution might be very non-linear. In other words, small changes to the inputs might lead to wildly different solutions. Think about the weather on Earth and you'll have an example of that. Set that aside for now, though. The information the planners need can't EVER be completed even in tiny family settings, but it might be possible to get close enough if the solution space isn't too non-linear. The problem with completion is that it is quite like a person's preference is unknown to them until they are faced with a decision and options from which to pick. They learn OF their preference by the result of their decision. Without a time machine, there is no way to get that preference back to a planner. 

How can a planner know the preferences of their family members when no one CAN know until the preferences are discovered? What we usually do is choose for them in advance and hope for the best. What do you want to eat for breakfast ten years from the day you first read this? Do you know? Why would you bother figuring it out, let alone recording it for someone in your family to help plan for it? You might be willing to plan a week ahead or do simple budgeting to help, but that is potentially useful. Why would someone in a paleolithic setting bother doing that? They couldn't be sure they would survive the next winter, let alone eight or nine more.

The second problem is  a mathematical one and is FAR from obvious. It is related to the fact that multivariate optimization problems become hideously complicated when the number of variables grows large. It gets so bad that the odds of finding a solution drop to near zero even if the number of possible solutions is finite or constrained as we expect they might be. I'll save this for next time, though.  In a nutshell, though, you need a time machine AND omniscience to pull it off yet mortal humanity does moderately well in finding solutions. If you write algorithms to automate problem solutions as tasks, be prepared to face your limits.

Saturday, February 23, 2013

For the Cynics: Unobtanium & Obtanium

I used to work on space-related projects that drew considerable skepticism from my peers. If any of them read this, they know EXACTLY to what I'm referring. For everyone else, don't sweat it too much. My friends who worked those projects with me were not so skeptical and we usually enjoyed ourselves in the work we did. We didn't always (ever?) accomplish what we set out to do, but we did learn other useful things along the way.

In the last project I worked, though, one particular skeptic framed his concerns well. He asked for a special explanation from me. If I was to avoid wasting his time in a pitch, he wanted to know precisely how I intended to accomplish what I said I could do. It was obvious to me he believed I couldn't. He was more than a skeptic; he was a self-admitted cynic. I was honest enough with myself to admit he could be correct too. I pondered his discussion requirement for awhile and then decided not to pursue him because I was fairly sure I would fail to convince him. 

Along the way, though, I worked up the terms and descriptions I include below. They are an expansion on the tongue-in-cheek terms obtanium and unobtanium. If you are unfamiliar with those terms you obviously haven't tried to build rockets and spaceships and convince people to invest their cash and time in your ideas. Don't worry about it, though. It is enough to realize that obtanium includes the first three entries below while unobtanium includes the last two. If you have tried to do such a project, though, I invite you to think VERY carefully about what you need to build what you have in mind or deliver the service you think can be brought to market at a profit before explaining any of it to me. I'm not the cynic my compatriot is, but I am decidedly more skeptical than I used to be.


Cotsium: (COTS-ium)

This is the stuff we can buy in retail outlets. It includes goods and services that share a common experience when one buys them. If one can pick them off a shelf and carry them to a check-out clerk, they are cotsium. If one can pick a service from a menu of options and pay a market price, it is cotsium. For example, internet access is a service we buy. It is cotsium today, but 30 years ago it wasn't. There are places on Earth where one cannot yet buy it, but that doesn't change the fact that is is still cotsium. If the market provides it in some places and not others, that is simply a demonstration of the profitability of the stuff being sold.

The boundary between cotsium and specialorderum is a little fuzzy, but here is a guide to help know where the line is. The key is to know the acronym. [Commercial Off-The-Shelf = COTS]

  1. Secret menus at restaurants still list cotsium products. Just because you have to ask to have your burger made with a veggie patty instead of beef doesn't mean you aren't buying the burger retail. The vendor might refer to your request as a special order, but they are just trying to make you feel better and return next time you want to had over cash to them.
  2. If you are buying a service from a small business owner and that owner pauses before quoting a price to you, it is possible you stepped over the line and are no longer buying cotsium stuff. The key to knowing is to figure out if the business owner knows how to price the service they provide to you. It is easy to mistake their pause for this, but be aware that they may simply be thinking about how to shake as much money out of you as possible.

SpecialOrderum: (Special-Order-Um)

This is the stuff that we can buy from craftsmen, artists, and other talented people, but it is generally sold in low volumes or involves unique events so no one including the seller knows how to price it at first. If you hire a photographer to take pictures of your kids, the photography service is generally cotsium, but if you extend the request and ask them to help make your kids look really good because they are competing as a team in a talent show the extension is a special order. The photographer might not know how to price that service at first, but will usually settle for charging for their time as much as they think they can get from you.

The key descriptor for specialorderum is the 'Um' people think or say as they try to figure out the price they are willing to ask or pay. We all know what it is like to ask for special things, but when the market volume for such a thing is small, the duration of the trade can become quite extended. Those with little experience bartering might not even know how to do it.

  1. If your kid asks for help on their homework or science fair project, they are asking for a special order from you. How you price it is up to you, but don't imagine for a moment that they won't remember the trade later when they want another special order and have something to compare.
  2. If you want some warm, fuzzy art you can buy a Kincaid print. It is cotsium. If you want a flattering portrait of your wife to make up for something senseless you said the other day, that is specialorderum and you'll pay dearly whether you get the painting or not. Your choice is how you pay.

NotYetium: (Not-Yet-ium)

This is the stuff that we can't buy yet, but we know it is coming. When we can buy it, it might be a special order or cotsium depending on the situation, but for now we can't get it and anticipate that we will soon. The new, spiffy smartphone you want with the next, spiffy feature and OS upgrade might not be on the market until summer, but you know it will be available because the vendor produces these upgrades like clockwork. You put yourself on a waiting list and stand outside the vendor's store the night before it is released. When you get it, you might bargain with their support staff to get your information migrated from your now dull and uninteresting smartphone to the new one. For a small fee they do it. They might even accept your old one in partial payment as a trade-in, but the amount they will give you depends on how well you have treated it. Condition matters after all.

Notyetium is wonderful stuff if you are a marketing person. You can sell it long before your company produces it and use that cash to finance your operations. If your customers aren't willing to hand over deposits to get on the waiting list, you can still borrow at reduced rates and pay off the loans with with the revenue you raise when your notyetium finally arrives. Do it well and you probably don't have to borrow much money anyway.

  1. A shipload of spices brought back by the Dutch to 17th century European markets was notyetium until it arrived in port. A shipload of gold taken by the Spanish from the New World at about the same time was notyetium until the ship arrived safely in port without getting raided by the Dutch or English. A CPU with double the transistor density compared to the one in your cutting edge computer is notyetium. Moore's Law tells us roughly when it will become cotsium or specialorderum depending on just how fancy your system is.
  2. If you think you can build a new, improved device and have experience building others like them, it is possible your new device is notyetium. If you have little experience, though, it probably isn't. If your device involves extracting free energy from the universe, for example, the odds are pretty high it isn't notyetium even if you think it is. If you want to buy Yeti fingernail clippings, though, I'm sure someone somewhere will figure out how to get them for you. Gullibility is definitely cotsium.


This is the stuff no one can get you, but no one can explain why it CAN'T be acquired either. It is a special category of magical stuff where no one can adequately argue the magic can't or won't be understood some day. Two hundred years ago our computers were unobtanium, yet we had the root knowledge for the mathematics that helps to describe them. We had rudimentary knowledge of algorithmic thinking and working examples of programmed looms to weave complex patterns into cloth. A solid-state CPU was unobtanium, though, for the simple reason that we didn't have the physics and engineering knowledge let alone the experience to build them. The magic needed wasn't yet known, but we know it now.

Unobtanium is really quite special. It is the stuff of dreams that aren't necessarily romantic fluff. We might learn later that the necessary magic isn't possible, but for know we don't know. People find this stuff to be quite motivating and even if the magic fails, they might move mountains in pursuit of it. Romanticism is powerful stuff.

  1. If you plan to put together a new website to provide social media services, you might be motivated by a dream to do it, but your product isn't unobtanium. We know most of the magic necessary to make a website work. You might argue that we don't know the magic that makes such a site successful, but I'll argue that it isn't magic at all. You just have to do the most difficult thing imaginable. You have to serve a useful function to others AND get them to notice you AND do it all for a price they find palatable. Good luck with that. Do it right and you'll be rich. Do it wrong and you'll be like everyone else because you'll wind up using the product your competitor made better than you did.
  2.  If you plan to take tourists to the surface of the Moon and back at a profit, you might be motivated by a grand vision of humanity expanding into space. Unless you know most of the technical details involved in such an effort, though, you need unobtanium before you'll make a profit. If your investors have a lick of sense, they will know that too. Good luck with that. Do it right and you'll have FAR more than a service for tourists. Your investors will know that too and will probably focus their questions on everything else besides your vision of tourists on the Moon. You might think you are in the space tourism business, but you are really in the R&D business.

Fantasticum: (Fantastic-Um)

This is the stuff no one can get to you and anyone with a modicum of education can tell you why it can't EVER be done. Your critics might be wrong, of course. To err is human. To engineer is human too. Unfortunately, there is a class of unobtanium where we are pretty sure the necessary magic is not possible. If your visionary product or service depends on such stuff, you need fantasticum. Basically, you need your fantasy to come true before you can succeed. Remember those Yeti fingernail clippings?  Maybe you have an idea for how to fuel a warp drive with them. The stars are within your grasp if you can just find a cooperative Yeti.

This is the realm of the dreamer who doesn't understand the physics well enough to know what can't be done. This is where you will find people who want to extract energy from the universe for nothing to build their utopia. This is where you will find the readers of science fiction who believe just a little too strongly that the stuff they read about in a story can surely be built. This is where you will find many of the romantics. They are the people who think that the world will flex for them if they can just find the right incantation and occasionally they are correct so it is easy to confuse them with people who pursue unobtanium. The key difference is that most everyone KNOWS the magic for fantasticum can't be done.
  1. Prior to the time the Chinese discovered the recipe for gunpowder there were many known incendiary and burning devices. Try to imagine those early days and the research done to find the recipe. It is thought the person who funded the work was after immortality and his alchemists tripped across a different kind of incantation. Mix the right ratio of sulfur, salt peter, and charcoal and you get an explosive. How magical is that, hmm? To the alchemists, gunpowder was unobtanium because they could believe the magic was possible and had no reason to believe otherwise. It turns out they were correct. Immortality through alchemy, though, appears to be fantasticum.
  2. Known science gets revised now and then. Because of this the border between unobtanium and fantasticum moves too. What we think we know to be impossible is difficult to nail down, but ask any scientist when they have had one beer too many and aren't overly worried about being precise and they will tell you. Most of them will agree on most of it too. Being tipsy only stops them from offering up their usual self-skepticism. If they still say they don't know something after two beers too many, maybe they really don't. To the less educated person, though, the border can be wished to be just beyond where they are now. No beers are necessary for that. If your work is like that, enjoy your romantic vision, but I'll go find some other project in which to invest my money.

Friday, February 22, 2013

Perceptual Blindness

Over the last few years I've been pushing myself to be involved in communities I have not typically found all that attractive in the sense that they aren't associated with my usual interests. These include a forum associated with the city I used to live in before 2009 and participation in discussions that show a break from my politically passive past. Most recently I've been getting involved with a political third-party and after the 2012 election I re-registered. Normally, I kept such outside activities associated to space-related efforts and the occasional physics/math discussion. My interest in those older topics is still high, but I haven't been putting as much time into space activities lately for a number of reasons that aren't terribly important. I'm sure I will return to them before long as I feel the pull to do something meaningful again.

What I want to put down here, though, are some thoughts about perceptual blindness. This is the kind of blindness where a person can be looking right at a thing and not see it because they don't expect it to be there or even exist. Imagine yourself in ancient times when people thought the Sun went around the Earth. The evidence we currently accept for the notion that the Earth goes around the Sun instead was right before their eyes, but because they already believed otherwise, they could see the sunrise and sunset as evidence the Sun went around us instead. The models we have in our minds that explain reality ARE how we perceive reality. Literally. If upon observing a sunrise and sunset the thoughts evoked in my mind are of geocentric astronomy, then I perceive the Sun going around us. If they evoke heliocentric astronomy, I see the Earth rotating in my mind and my view of the sky shifting as I move with it. The models we make in our heads and teach to our kids have more to do with our truths than our sensory data does.

Perceptual blindness and other related issues have been studied in terms of optical and auditory illusions for some time. There is quite a pile of evidence now pointing to the fact that we must also have senses that point inward at the models we construct as we learn about the world in order to know when the external sensory data triggers one or more of the models. My experience in this area has less to do with illusions, though, and more to do with alternate narrative explanations. Ponder this scenario.

Late at night I wake up to the realization that I left my computer monitor on in my office. It has gone black, but not powered off so there is a mild glow coming from the room that I usually find annoying when trying to sleep. I get up to shut it off, but I don't want to turn on any lights and feel the pain of the glare and loss of my dark adapted vision. I pad into my office using the low glare from my monitor as lighting. When I'm almost there and about to reach over to push the power button, I step on something that is wet. The entire path to my desk is carpeted where I live and the first thing I conclude is that the dog has pissed on the carpet in front of my desk again as a way to retaliate. That thought arrives in a flash along with the anger and disgust when I realize the carpet might not JUST be wet. The question is, can I reasonably conclude that the dog did that? Can I conclude the dog did more? Should I consider alternative options before getting angry?

Anyone who has found themselves in a similar situation knows that rational thought never gets even the slightest chance to intervene. There simply isn't enough time. The wet feeling between my toes matches a previous experience where I do know for a fact that the dog peed on my part of the carpet. I've learned to close my office door to prevent that option and the dog goes elsewhere when his tiny little bladder isn't large enough to make it through the night. He is a chihuahua too, so I really shouldn't blame him for having such a tiny bladder, right? He is what he is and could very well be trying to find the most out of the way place to do the deed. My office certainly qualifies since the door is rarely open for him anymore and when I'm at my desk he is a bit too scared to come in. None of that matters though when I match in a flash the narrative that the dog is retaliating for my pressing my will on him when it comes to establishing dominance in this house or during walks and all that. I don't even know if that makes any sense in the dog psychology way, but I don't think about any of that in the brief flash before anger and disgust.

What I find interesting about this is that in political communities I always advocate for a calmer interpretation of events. I always push for an exploration of alternative narratives even if they contradict ones I like and prefer. I'm not perfect in this as I am a little less inclined to explore alternatives to interpretations I like. I suspect most people do that, so I don't feel guilty about it. I learned to think this way after getting my lip busted in high school. I thought I knew what was going through the other kid's head and failed miserably to anticipate his level of anger. The truth was obviously a closer match to an alternative narrative I had not considered. Over the years I realized it was an alternative I didn't WANT to consider and in self-defense I learned to squash that anti-want.

Let me bring this story back to the present, though. I was watching a presentation the other night by an author with a strong libertarian view of life. He said a number of things I found to be agreeable and a few that I thought were quite bizarre. He spoke of life under the thumb of our government and while I recognize the risk of such a future, I obviously don't see things as he does. What I got to wondering is whether it was him or me that was perceiving the world in terms of a geocentric astronomy. He tried to articulate some of the threat he saw, but didn't get far because the other libertarians just nodded with him making it rather clear he didn't have to explain it to them. I let it go for further study later as I didn't want to interrupt his talk. It was an odd experience for me.

In the forum I frequent, though, it is often the case that the shoe is on the other foot. I wind up seeing potential threats to liberty where others do not. I'm not quite the lone nut case preaching doom and the end of the world. There are a couple of others who are close enough that they nod occasionally, but again I wonder who is the fan of geocentric astronomy. It is still fun to debate with all of them. I get a chance to have my ideas beaten up and completely thrashed instead of suffering that pain upon my body directly. I like to think my ideas have improved as a result. I've had to face some of my own mental dissonance that only another person can point out and it has been useful to me.

What I find most interesting about these related lessons, though, is that I know I can be perceptually blind. I know I can flash to a narrative explanation for events whether it involves dogs in my office or politicians taxing me and establishing competing services to what I would like to do. I know other people can point out my error if they do not use precisely the same perceptual model I use. I also know I can return the favor when they are in error. The problem, though, is that I know I am occasionally correct and the other person wrong when they believe the opposite. Both of us can be perceptually blind, thus there is no formal way to decide who has the most truthful narrative. Even a vote taken among a large group of people noodling over the same problem and evidence isn't enough to formally decide. I don't think there IS a way to know absent a metaphysical observer with omniscience. Even then that wouldn't work since my perceptual model of the universe has no room for such an entity. I simply wouldn't believe them, thus I wouldn't see the truth they offered.

Ultimately, I think this is why I have to defend liberty. If we can't decide who is right and can't agree on what to do, we have to tolerate each of us doing as they wish and letting time prove us right or wrong. Time might not oblige us with a proof, but when it does it is usually pretty obvious.  If someone tells me it is perfectly fine to talk on their cell phone and drive at the same time, I know as a last resort I can just wait and watch. My perceptual model of the risks says they won't have time to think about the danger they will eventually face some day in a complex encounter on the road. Eventually the paramedics will be called to scrape them off the highway along with other innocents who unintentionally helped prove what a stupid idea that was. Ultimately we prove our truths with our lives, but not everyone can see that truth either.

Monday, February 18, 2013

Physically modeling the ambiguous and the undefined

When we try to reduce the complexity of the world around us to understandable parts and relationships we look for patterns and then model part of the world as if that pattern explains what is happening. If I toss a rock in a pond and notice ripples on the surface, I explain the ripples in a causal manner with some kind of narrative that requires rocks thrown into ponds to lead to ripples. With such a narrative, I can try to work backwards from an observation of ripples to a statement that somehow a rock was tossed in, but there are the usual risks with working backwards with a model. There could be some degeneracy in the effects of a wide range of causes.

In the language of mathematics as it is used by physicists, we write equations for the energy and momentum of the rock and bits of fluid. We write equations for the behavior of fluids and rigid bodies tossed into them. We also use a few trial runs with the combined model to see if it works and tweak internal parameters until the predictions the model makes match well enough with reality. This kind of modeling can be done on paper or on computers by someone with a little bit of training, but at its most basic it is pretty simple. Until a model gets complicated (and they do... very fast), the mathematics doesn't get much worse than differential equations, geometry, algebra, and linear algebra. If you don't know some of these, don't freak out. You've probably used them informally and didn't know it though your experience might have been painful.

Whether one is comfortable writing physical models or not, there are some assumptions that go into them that don't involve fancy math. For example, if one models the rock tossed into the pond using momentum and energy, one has to assume the rock HAS momentum and energy, right? Energy comes in more than one form, so we might associate a few different numbers with the rock and expect them to change as it flies to its collision with the water's surface. Momentum is treated as a vector which is a number with a direction. We assign a momentum property to the rock too and expect it to change in flight. How much precision we expect for energy and momentum depends on how accurately we try to measure the flight, but we expect the rock to have well defined values for each. That is required to model the rock, right?

It turns out that there are alternate versions of these assumptions. If one models the rock as described above with well defined momentum and energy one is guilty of Classical Thinking. What this means is that we assume the rock has these properties AND that they are defined independent of anyone actually measuring them. Running the equations forward or backward in time to describe the flight path and collision speed has the built in assumption that the attributes exist. This assumption turns out to be incorrect, but physicists didn't grapple with this fact until the 20th century. On top of that there are a whole class of problems where we can safely make this erroneous assumption and still get good predictions from our models. It is as if we try to predict the positions of planets in the sky using Ptolemy's geocentric model of the universe and get reasonably good answers. The explanatory part of the model is wrong, but the rest of it works to deliver accurate results that appear not to depend on the bad assumptions.

What other assumption could be made besides the ones from classical thinking? This is where quantum thinking comes from and there are two possible versions of which I know. The first is the one we adopted and we've only recently discovered it too is wrong. The second is simply bizarre. 

One break with classical assumptions is to permit the rock not to have well defined momentum and energy, but it DOES have momentum and energy. This is Early Quantum Thinking and means that a question that reads like 'how much energy does the rock have at a particular time' is not answerable absent some kind of measurement even theoretically. We must do something to the rock to discover the energy and until we do it the answer is ambiguous. That means one can't 'just run the equations.' The rock could have several values for its energy and still be what it is. It doesn't have one value until we force a situation where it must. 

If you know any philosophy this should bother you because it comes very close to saying the universe is what we say it is because of what we do to it. No reasonable person trained in these quantum models would actually make that claim, but the untrained reader often makes the leap to thinking they can imagine something and make it so. There are so many who will make this leap that an unscrupulous author can make quite a bundle selling this nonsense. Those with a stronger sense of moral duty to their community will instead speak of Schrodinger's cat because that little thought experiment nicely explains the oddities that come from this way of modeling. 

Unfortunately, though, it appears this is all wrong. This break with classical thinking still assumes that momentum and energy have meaning independent of observation. We model the rock as a mixed state of possible values and let an operation sort out which one actually occurs. The operations are probabilistic and would satisfy any Las Vegas bookie as impossible to fix, though one might sway them in understandable ways.

There is another way to break with classical thinking and it appears there is good experimental evidence that the universe works this way. Modern Quantum Thinking takes the next step and breaks the assumption that the attributes we measure have any meaning independent of the operation that measures them. An operation that measures the energy of the rock produces a number AND the meaning for that number. The rock itself doesn't have energy until it is measured in a way that would answer the question that asks how much energy it has. It is not that the rock has zero energy, though. It is that the energy concept is undefined. It is as if a programmer wants to read from a variable and forgot to declare it in their code earlier. The operation that answers questions about energy declares it and then measures it in one swoop. THAT is how the universe appears to work.

A whole set of reasonable questions arise now. How does one model an attribute that has no meaning until the object being modeled is measured in a way that produces the meaning AND value? How does one test such a model against reality? Can one distinguish between a successful model and one that actually explains reality? Remember the difference between geocentric and heliocentric models of the solar system. Both can be made to work, but one is very wrong! Can one write falsifiable models that work this new way? The ability to falsify models is critical to the health of science.

Modern quantum thinking can break a different assumption not mentioned above as well. We could break our assumption that phenomenon are separable. If one measurement is taken in a distant corner of the universe and another taken here, they can't influence each other as there isn't enough time for information from one to get to the other. That too appears to be wrong, but it is unclear whether this break is required or the break with meaning is required. One thing we do know is that we must either surrender separation (localization) or attribute meanings until operations occur. One of those is simply wrong and maybe both.

Is it all clear as mud now? Welcome to the world of physics. This is what some of us are pondering right now. The experimental physicists are busy in ways that occasionally hit the news sites (LHC work discovering the Higgs particle for example), but the the theorists are plugging away at the weirdest stuff you can imagine... or maybe you can't. I'm not sure I can because I'm trying to figure out how in this world I can model undefined attributes. Breaking localization doesn't bother me much, but attribute meanings too? That's asking a lot.