By Paul Almond
This is the first in a series of articles exploring the relationship between minds and physical systems (substrates) on which they are based. Strong AI (strong artificial intelligence) advocates typically maintain that the substrate is irrelevant, provided that the required computation can be performed on it, and only the computation matters. John Searle, an opponent of strong AI, argues that the substrate does matter and that a mind is not just computation on a substrate but is caused by specific physical processes. Searle states that there is no reason to assume that all substrates that allow general computation can support minds [1,2]. This article will show that the substrate matters, but not in the way that Searle thinks. It influences the probabilities that you are in various situations in some thought experiments in which there is uncertainty about the substrate on which you currently exist. The substrate is statistically important and influences the measure of minds associated with computing done on it. How this may relate to expectations of future status and influence ethics, in terms of the value that we assign to a thinking entity, are explored. There is an apparent paradox between the idea of associating measure with minds that emerges from the thought experiment in this article and our experience of seeing single instances of intelligent entities in the real world. This does not mean that we should avoid the issues raised by the thought experiment, but we need to consider how minds are associated with substrates to resolve it.
The second article will show how a many-worlds type view, arrived at by considering what the existence of objects means, resolves this apparent paradox and how the concept of measure can be meaningful when applied to a mind associated with a substrate.
The third article will discuss some implications of the arguments made previously.
The fourth article will consider the implications for strong AI and Searle's argument against it in particular. Searle's case relies on the nature of a substrate being important regarding whether or not it can support a mind and on the irrelevance of the substrate in strong AI. It will be shown that admitting statistical relevance of the substrate, or its effect on the measure of minds, does not discard so much of the idea of irrelevance of substrate that strong AI fails, but partially giving up irrelevance of substrate like this does not help Searle's argument, while it does, in fact, allow strong AI to be more clearly expressed in a way that allows Searle's arguments to be more easily dealt with. Strong AI's claim that different substrates can support minds is generally correct, although what is meant by "computation" needs clarification and the statistical importance of substrate should be acknowledged.
Introduction
A widespread view is that human minds exist due to computation in brains. The brain is a physical system providing a substrate on which the mind is based. Computation can occur on different substrates; for example, electronic or mechanical computers.
Some proponents of a philosophical position known as strong AI think that the substrate is irrelevant and, provided that the correct computation is occurring, a mind exists. This article will consider the importance of the substrate supporting a mind. A thought experiment will show that the substrate is not completely irrelevant, but has some statistical importance. The nature of a substrate influences the chance that you are based on it if you are uncertain about your status and, by implication, the measure of minds based on it.
Strong AI and Substrate Independence
The term strong AI (strong artificial intelligence) was given by John Searle to the proposition that computers can be conscious and he described it as follows:
"...according to strong AI, the computer is not merely a tool in the study of the mind; rather, the appropriately programmed computer really is a mind, in the sense that computers given the right programs can be literally said to understand and have other cognitive states. In strong AI, because the programmed computer has cognitive states, the programs are not mere tools that enable us to test psychological explanations; rather, the programs are themselves the explanations." [2]
The strong AI case would seem to imply that:
- consciousness can be viewed as existing if symbols are being manipulated in the right way by a physical device.
- consciousness can be associated with the behaviour of a system. If a system is acting in the right way, as determined by observations of its external behaviour, then it can be viewed as being conscious, or as one scientist casually put it, "If it walks like a duck and quacks like a duck, it is a duck."
- if a system were conscious then a properly constructed model of that system would also be conscious, irrespective of how it was physically realized. Computers can be built from semiconductors or wooden rods and string. The strong AI case would say that this choice of substrate is philosophically irrelevant.
Searle gives the term weak AI (weak artificial intelligence) to the position that computers are capable of modelling the behaviour of conscious entities. Weak AI differs from strong AI in that it does not regard such computers as necessarily conscious merely by virtue of them appearing to behave in the same way, to an external observer, as systems that are conscious.
Strong AI asserts that the nature of the substrate is irrelevant to the issue of whether a mind exists on it or not, provided that the necessary symbol manipulation can be done on it. From this, an obvious implication, although not one necessarily stated as part of strong AI, is that the nature of the substrate has no philosophical relevance whatsoever. The thought experiment in this article will show that the substrate has statistical relevance. Although this may require the strong AI position to be revised slightly, this sort of relevance is too weak to help Searle's case that strong AI is wrong.
Ideas Behind the Thought Experiment
The thought experiment will use two main ideas: mind uploading and uncertainty about your current situation.
Mind uploading [3,4] is the idea of using a hypothetical, very accurate brain scanning procedure to construct a digital representation of a human brain in a computer. This is then used to run a simulated model of the brain to make a digital "copy" of the person who was "uploaded." Mind uploading will form the basis of the thought experiment, but this does not mean that it must be considered practically plausible: it is merely being used as a philosophical device. Nor need we worry about whether a mind uploaded copy of you would be a continuation of your identity: the thought experiment assumes that you already exist as a mind uploaded entity. The issue of continuation of identity is tentatively explored later in the article. This article is not really about mind uploading, which is only being used to create a scenario in which you are unsure about your current situation. The type of mind uploading imagined to be occurring in this article requires you to be asleep while it happens, to make things convenient for me.
The uncertainty about your situation which will be used in the thought experiment is uncertainty about the substrate on which you currently exist. That is to say, you are unsure about the nature of the physical system which is causing your mental experiences. Most people probably are quite sure about the nature of their own substrate: it is their brain, if we ignore Nick Bostrom's simulation argument [5,6], to which this series of articles has some relevance, incidentally. It is possible, however, to imagine situations in which you are less sure. Greg Egan's novel Permutation City [7] has situations in which characters are unsure. At the book's start a character is unsure whether he is an organic person or a "copy" made by mind uploading running on a computer in a virtual reality. The thought experiment will use the same sort of idea, with a scenario in which you could be in a number of different situations, existing on any one of a number of different substrates.
The Thought Experiment
Imagine finding yourself in the following situation:
You wake and see a window floating in front of you containing the text "You are one of the copies." Your last memory is of being in a reclining chair, about to go to sleep to undergo a brain scanning procedure which was to be used for "mind uploading" -- making a computer simulation of your brain. Three identical simulations of your brain were to be run on three different computers: A, B and C. A, B and C run identical software and start in the same state. Each of the uploaded copies was going to find itself waking in a virtual reality with a window apparently floating in front of it containing the text "You are one of the copies." You know that you must be one of the copies: you cannot be the original, organic person. When this is happening to you, none of the simulations in A, B and C, after starting off in the same state, has yet interacted with the outside world. Although such interaction cannot be ruled out in the future, it does not matter now.
You are not told on which computer A, B or C you are actually running. You know that, whichever it is, two more versions of you are running on two other computers, each equally uncertain about his/her own status. As far as you know, there is still the original, organic version of you in the real world, unless something has happened to him/her but this person is irrelevant here.
How likely is it that you are in Computer A, B or C? You have no way of differentiating between them, so the probability is 1/3 in each case.
We now cosmetically change the computers by removing the exterior casing from computers A and B, enclosing them both in a single, larger case, so that from outside it appears to be a single computer. Internally, nothing has changed: this new box simply contains two computers, A and B. Although we appear to have two computers in the box, they are both doing the same processing and we really have a single computer with some redundancy, so we will call this new box "Computer AB."
Computer C is unaltered: it remains in its original case and will continue to be called "Computer C." The probability that you were in Computer C was originally 1/3. We have merely altered some exterior casing and it is unreasonable to expect this to affect probability.
So, the probability you are in Computer C is still 1/3.
You will be in Computer AB if you are in what was originally called Computer A or what was originally called Computer B, which are now inside the box called "Computer AB." The probability is 1/3 for each of these possibilities.
Probability you are in Computer AB = probability you are in Computer A + probability you are in Computer B
= 1/3 + 1/3
= 2/3
We have two computers with different internal workings and:
Probability you are in Computer AB = 2/3
Probability you are in Computer C = 1/3
The probabilities are different for these two computers.
If you object to Computer AB being regarded as a single computer, saying that it is two computers labelled as a single computer in a contrived way, then imagine that the computers are mechanical and made of metal components. Imagine sliding the two computers together, closer and closer, so that they overlap, until equivalent parts are adjacent and can be welded together. Alternatively, imagine two electrical computers of some kind which are moved together to make a single computer with thicker wiring, or two electrical computers with lots of gaps between atoms allowing us to merge the machines together by overlapping them so that they fill each other's gaps. Even though it may be practically implausible we should be able to imagine, philosophically, various merging processes like this, each producing what can only sensibly be considered a single machine. Would it be reasonable to say that the probabilities for computers A, B and C, or the combined probabilities for any group of such computers, must change appreciably during such a merging process? Is this supposed to happen magically in the instant when equivalent components touch each other?
If we get different probabilities like this for computers working in essentially the same way, just by altering the amount of redundancy or merging components together, it follows that more extreme changes to the substrate, such as completely altering the design of the machine or its working principles, can also cause a change in the probabilities.
Therefore:
If you are in one of a number of different situations which involve you existing on different substrates, then the nature of each substrate affects the probability that you are running on it. The substrate is not irrelevant, but has statistical significance.
Substrate and Redundancy as a Placeholder
In the thought experiment, and similar ones that we may consider, the feature of a system that makes it particularly likely that you are inhabiting it is redundancy. Computer AB is a substrate that provides more redundancy than Computer C because it can be used to run two versions, each equivalent to Computer C. It is not just "how many computers you have in the box," though. When the individual computers inside the case of Computer AB are moved closer together, being ultimately combined, it may not be immediately obvious that redundancy still exists, but it does in one obvious sense: Computer AB is made from twice as much matter as Computer C, making it equivalent to two versions of Computer C which have been merged. The redundancy, therefore, relates to inefficient use of matter. Computer C will also have its own redundancy, but when we are considering the probabilities it is the relative redundancies of Computer AB and Computer C that matter.
Many aspects of a computer's design relate to its redundancy. In any computer, duplicates of components would mean more redundancy. In an electrical computer thicker wires would seem to involve more redundancy, as would larger gears and levers in a mechanical computer. If two computers based on the same principle, such as two mechanical computers or two electrical computers, can have different amounts of redundancy, then when the operating principles are different there is even less reason to expect the redundancy to be the same. We should not expect the redundancy provided by a given electronic computer to be the same as that provided by a given mechanical computer and we should not expect the redundancy provided by the human brain to be the same as that provided by various other types of system.
In scenarios like the one in the thought experiment, substrates which work on different principles could have very different probabilities.
The argument is not claiming that redundancy is the only characteristic of a substrate that determines probabilities like this. Redundancy merely tells us how we should assign probabilities all else being equal. Situations in which you might exist could have other features which can be shown by different philosophical arguments to affect the probabilities. This "all else being equal" qualification applies throughout this article.
Some readers may have issues with this "redundancy" concept because it is hard to formally define, and they would have a good point, but this is not the end of the matter. "Redundancy" is a cruder version of a more sophisticated explanation of what is going on that will be given in the next article. This more sophisticated idea is based on numbers of different ways of algorithmically extracting patterns or meanings. For now, however, "redundancy" is a useful "placeholder" idea.
Considering Organic Brains
The thought experiment and argument just given relates to organic brains as much as anything else. In the scenario just given you knew that you were the uploaded copy, but what if you did not know? Imagine this thought experiment:
You wake in a chair in a brain scanning room. Your last memory is of being about to go to sleep for a scan. You had arranged that, after the scan was made, a number of uploaded copies of you would be run on various computers. Each of these copies would be presented with a virtual reality simulation of waking in the brain scanning chair with no immediate clues from the environment that it is no longer the original, organic version. There is no way of knowing, just by looking around, whether you are the original, organic person waking in the chair after the scan or one of the uploaded copies in a virtual reality simulation of waking in the chair. Maybe, of course, the simulated environment has limits. If you are simulated and try to leave the room, the building or whatever city you are supposed to be in, then maybe you will find it impossible, for example (as happens to a character in Permutation City [8], but this need not concern us: right now, you do not know. How likely is it that you are the original, organic person?
If each version of you is as statistically important as any other then the answer is simple: there is one organic version of you and two computer simulations, so the probability that you are the organic version is therefore 1/3 and the probability that you an uploaded copy is 2/3. The argument previously given about substrate importance, however, throws doubt onto this. If the contents of the computer cases were previously important in the thought experiment, then they are important now. We cannot just count versions. Instead, we need to look at the substrate in each case. We need to consider the substrate for the organic version, which means considering the physical nature of the brain itself, and we need to consider the substrate in the case of each of the computers running uploaded versions. The more redundancy in a particular implementation of a version you, then the more likely it is that that implementation is causing your experiences.
Expectation of Future Status
An argument like this about your current status could possibly be extended into a tentative argument about expectation of future status.
As before, imagine that a scan will be made of your brain and used to make a number of copies, each of which will be presented with a virtual reality simulation of waking in the brain scanning chair with no immediate clues from the environment that it is no longer the original, organic version.
On waking in the chair you will be unsure whether you are the original, organic version or one of the copies. You will only be able to assign a probability to each possible situation and, from the previous argument about substrate and probability, the types of substrate will affect this probability.
If you can determine the probabilities when apparently waking in the chair, just from considering different substrates, you can as easily work out the probabilities that you will later assign to different situations on waking before the brain scanning occurs.
As an example, let us imagine that, before the brain scan, you decide that when you wake in the chair, with no sensory information available to resolve your possible situations apart, you will conclude that there is a 60% chance that you are the original, organic version. This means that right now, as an organic person, you think that in the future you will think that there is a 40% chance that you are the copy. Maybe, if you trust your reasoning to be valid after the mind uploading procedure, you should assume the same reasoning to be valid now when you use it to determine your expected future status? Maybe you should assume, now, that after the mind uploading procedure there is a 60% chance that you will find you are the original, organic version and a 40% chance that you will find that you are the copy?
We do have the issue, here, that we are assuming that you do not have any experiential way of resolving the situations apart. What if such experience were available? Maybe, for example, you would expect to be in a different room if you were the uploaded copy? Either this is a reasonable argument about expectation of future status or it is not. If it is reasonable then it is implausible that your expectation of future status should vary depending on how many clues you are given from the environment. Saying that you allocate a certain probability to being the copy or the original accepts the possibility that you can find out later which you are if some experiential information becomes available. Even if experiential information starts to become available immediately (that is to say, there is no attempt to fool the copy), it will take you time to acquire and process sufficient information. On the other hand, if you have thought about issues like substrate difference in advance you may already know what the probabilities will be of finding yourself in different situations. No matter what experiential evidence is available, or whether or not it is available immediately, there will be at least an instant when you have no observational evidence on which to base any assessment of your situation and in which you will have to base your consideration on the issue of substrate discussed here and, possibly, other issues.
When multiple copies of you are going to exist, one of which could be your original, biological brain, a case can be made that you should view this as involving multiple futures and you should assign yourself a probability of finding yourself in any one of these futures. That probability would be determined by ignoring any sensory evidence that could be presented to you in this in these situations, assuming that you would have no experiential way of telling the situations apart, and considering how your knowledge about the substrate in each case would affect the probability you assigned yourself of being in each possible situation after the copying process.
This may seem a strange view. It would mean, for example, that if you are going to have a copy of yourself made from a brain scan then, from your point of view, before the scan is made, there is a chance that you will become the copy after it is made and activated. Readers may be sceptical about this, but it is hard to find a firm reason against it. Both the copy and the future state of your brain are effectively patterns propagated into the future by some causal sequence of events linking your current and future states. What I would be sceptical about is the idea that substrate is the only issue when considering your chances of finding yourself in any future given situation. No assumption should be made, for example, that the original, organic version and any simulated copies automatically have the same status until we start considering substrate differences. It is possible that what has happened in the past of a particular version, whether it has just relied on the biological brain continuing to exist or on various processes associated with mind uploading, does matter. The sort of consideration of substrate that we have been discussing would tell us how to assign probabilities all else being equal and other factors could play a part.
Objections can be made against the idea that all that matters is future uncertainty. One objection involves constructing a thought experiment in which we know now that one of the situations that we will consider possible in the future is actually impossible. This is the scenario:
You have volunteered to go and live in the Mars colony for the rest of your life and in a moment you will go to sleep in a hibernation chamber on a spaceship. When you wake up you will have arrived at Mars. You know that they have trouble getting enough volunteers to go to Mars and they get round this by manufacturing colonists in unused hibernation chambers on the spaceships en route to Mars, using advanced technology that can actually build a human being, atom by atom, from raw materials. The practical plausibility of this should not be an issue, but if you need an idea of how the manufacturing process might work, imagine molecular nanotechnology [9,10,11,12,13], as proposed by K. Eric Drexler, being used. Fake memories are implanted into the manufactured colonists as part of the manufacturing process: the brains are built with neuron wiring patterns containing memories. These memories are of a made-up life on Earth prior to volunteering to go to Mars, right up to the point of going to sleep for the journey. On arrival at Mars the colonists who actually lived on Earth and volunteered (whom we will call "real" colonists) and the colonists who were manufactured en route and only think they lived on Earth (whom we will call "fake" colonists) are all woken to colonize the planet. For every "real" colonist, one "fake" colonist is manufactured en route. There is no attempt to hide this process from the inhabitants of Mars. They are all told that half of them never existed prior to the journey and proof is made available that this is actually the case. We will not worry about what form this proof takes, but it is available on Mars. The colonists are not told, however, which of them are "real" colonists and which are "fake" colonists and no communication with Earth is permitted, so colonists cannot find out, for example, if people remember them on Earth. You know all this while waiting to go to sleep for the journey.
On waking at Mars you will be a member of a population of people, half of which are "real" colonists and half of which are "fake" colonists. You will not know which, however. In the absence of anything else on which to base your judgement, when you are on Mars you should think that there is a 1/2 chance that you are a "real" colonist with real memories of life on Earth and a 1/2 chance that you are a "fake" colonist with fake memories of life prior to waking at Mars. What makes this situation strange is that right now, before going to sleep, you know that you are one of the "real" colonists because, right now, you are actually experiencing life prior to waking up at Mars -- experience that manufactured colonists never really have. You also know that this knowledge will not help you when you arrive at Mars. Even though you know yourself to be a "real" colonist now, you also know that when you wake at Mars you will not know if your current experience of being sure is a fake memory and that you will still need to assign yourself a 1/2 chance of being a "real" colonist.
If knowledge of future probability estimates always indicates future expectation, then the knowledge that in the future you will assign yourself a 1/2 probability of being a "real" colonist and a 1/2 probability of being a "manufactured" colonist indicates that in the future there is a 1/2 chance that you will be a "real" colonist and a 1/2 chance that you will be a "manufactured" colonist -- even though you now know that you a "real" colonist. This does not seem to make sense: you can hardly change from a "real" colonist into a "fake" colonist. Where does this leave us? Substrate in different possible future situations was related to the idea that future uncertainty is significant in determining the probability of being in future situations, but this objection seems to throw doubt on this. My own view is that the objection may not mean that uncertainty is not worth considering, but rather than we need to be cautious about it and if we do use expected future probability values to indicate future expectation we should not expect this to apply in all situations.
We could make another, stranger interpretation of the Mars colony thought experiment. Instead of considering the memories of the "fake" colonists as being really fake we might decide that these memories are of real experiences associated with whatever computational process was used to generate the fake memories. In this view, there would be no such thing as truly "fake" memories because all memories would have to be generated by some process and your "fake" memories would really be of your experience of that process. If true, this would mean that, in the thought experiment, while waiting to go to sleep in the hibernation chamber, you could actually not be sure that you are a "real" colonist: you could be a fake colonist whose current experience is based on the substrate of the computer used to generate your "fake" memories or on the process of constructing your neurons with all the wiring to contain them, or even on the processes that occur later in your brain during recall of the memories. A feeling of this sort of idea is provided in Alastair Reynolds's novel Revelation Space [14] in which an implanted memory is described by narrative, as if the events were actually experienced by the character, though I am not sure if any philosophical suggestion is being made by this or if it is just an interesting, or convenient, literary device.
Ethics and Value
It could be argued that the probability of finding yourself in various situations like this is linked to the value that should be placed on things. As an example, consider the following thought experiment:
You are a simulated copy running in a virtual reality on one of two computers: Computer A and Computer B. Both computers are running identical versions of you and you are given technical descriptions of both computers, but you do not know whether you are in Computer A or Computer B. You receive information from the "real world" that both computers are at risk of attack by the HellSim V99.2 computer virus and you can assume that the other version of you has received similar information. This nasty piece of software "infects" computer systems running uploaded minds in virtual realities and twists the virtual reality simulation to cause great suffering to any inhabitants -- and that means you. The danger to each computer can be reduced by spending money on it, and the more money that is spent on a given computer, the lower is its risk of viral infection. You have a limited amount of money to spend in the real world and someone in the real world will follow your instructions about how to spend it to buy safety for each computer. This person will not tell you which computer is running you. You need to decide how much money to allocate to reducing the risk of viral infection for each computer.
You could spend the same on each computer, but that could be simplistic. If the two computers differ in construction -- if there is a difference in the possible substrates -- then the previous argument suggests that you could be more likely to be in one computer than the other and it may be rational to spend more on reducing the risk to that computer. The "all else being equal" qualification applies. Substrate may not the only issue -- there could be others -- but it is an issue as it affects probability. If you would spend different amounts of money on protecting different computers then you are assigning different value to the different computers. When considering computers that could be running you, value appears to be related to substrate.
If you would apply this reasoning to computers running copies of you, why should you not apply it to other entities? This suggests that if there is a computer running a thinking entity then the value placed on that computer should not be based just on the entity that it is running but on the physical nature of the computer itself. All else being equal, it seems that the greater the degree of redundancy in the substrate, the greater the value that we should assign to that substrate. Different computers running the same kind of entity, with the same behaviour, might be assigned different value, and different levels of rights, due to different physical construction. This is different from the view that many strong AI advocates have -- that the nature of the substrate has nothing to do with the rights that a machine should have.
I need to be clear about how this should be interpreted. I am not saying that intelligent computers should not have rights; as I think that human thought processes are computational it would not be sensible for me to suggest that. Nor am I saying that computers should automatically have less value, or fewer rights, than humans. The physical nature of a system is simply one characteristic that may play a part in assigning value. Other features, such as the nature of the entity being simulated, may be far more important in determining the relative value of different systems in many situations. I am also not suggesting that we need to deal practically with ethics in this way in everyday situations. Even if philosophical arguments can be made about the nature of a substrate and value, we may decide not to make fine judgements in all instances, instead choosing to assign the same value to all systems of a certain general type.
The Apparent Paradox
A paradox may seem to result from the conclusions reached so far. It has been argued that the type of substrate on which you could exist in different situations could influence your probabilities of being in these situations if you cannot resolve them apart, and that this can influence the value that we assign to physical systems "running" different thinking entities. This would seem to need multiple versions of you and other thinking entities to justify it, but this is not what we see when we observe systems externally.
As an example, returning to the thought experiment from the start of the article, you can be in one of three systems, A, B or C, and the probability of each is 1/3. That is consistent with what an external observer would see: he/she would see three boxes and each box, if analyzed closely, would appear to be simulating a single version of you. If there are three versions of you, each in a different situation, then, all else being equal, any one of these versions should assume a 1/3 chance of being in each situation. When we put Computer A and Computer B in the same box, making Computer AB, things could still be easily seen to make sense. There is a 2/3 probability that you are in Computer AB and a 1/3 probability that you are in Computer C. The 2/3 probability for Computer AB may seem strange to an external observer, who only sees one computer, but on looking inside Computer AB he/she can see that it is made of 2 computers, A and B, each running a version of you, and this could account for the 2/3 probability. Things get difficult, however, with two computers that have different probabilities merely by having different substrates, as when AB is made by merging A and B together more extremely. We could have a situation where there is a 2/3 probability that you are in Computer AB and a 1/3 probability that you are in Computer C, but when you look inside AB and C you do not see any "multiple versions," but merely a different substrate than that provided by Computer C. This substrate may be equivalent to multiple versions, but it does not change the fact that when you look at Computer AB or Computer C you do not see them: you see a box behaving in a particular way.
This is the apparent paradox: when we are considering these systems from "inside," when they are possible candidates for our situation, each system seems to correspond in a sense to many possible situations for us, yet to an outside observer there is just the system exhibiting whatever behaviour it is exhibiting.
This issue is most obvious when we mix value with it. Suppose we have some thinking entity running on two computers, A and B, which are both under some kind of threat. We can spend resources on trying to protect Computer A, Computer B or both. The entity being simulated in each case appears scared and loud pleas to be protected are coming from loudspeakers on both computers. What if the substrate provided by Computer A supported the entity with much more redundancy than that provided by Computer B? According to the argument given so far, we should give Computer A greater value than Computer B, and we should be prepared to spend more of our resources on trying protect Computer A, because Computer A and Computer B can be considered equivalent to multiple computers running versions of the thinking entity and the greater redundancy provided by Computer A makes it equivalent to a computer running more versions. On looking inside Computer A's case, however, we do not see all these separate computers laid out in front of us, doing redundant processing: all we see is an architecture that is less efficient, in one sense, than Computer B's. For example, we might see that Computer A has thicker wires, or uses a different technology entirely than Computer B. The collection of computers to which A is equivalent appears in some sense to be a philosophical device, introduced to deal with this issue of what happens when we move two computers progressively closer together, yet now it is forcing our ethical decisions. One fact is obvious: despite all our argument about probability, substrate and value, we do not hear more pleas to be protected from Computer A.
Observation and conventional intuition would suggest that there is only one entity in each computer. In fact, some people would say that the entity "in" each computer is just an abstracted way of describing the computer itself, yet the argument about substrate seems to suggest that when an entity runs on some substrate there is really a number of entities there -- that it has some kind of measure dependent on the nature of the substrate itself. How do we resolve this apparent paradox?
It may seem tempting to discard the earlier argument about substrate as an aberration or something only meaningful in some abstract sense, but I will not do that. We can get meaningful results for three separate computers. We can get meaningful results when we enclose two computers in the case. There is no reason for things abruptly to become an aberration, or abstract, when we start to move two of the computers closer together, ultimately merging them. The substrate argument will not go away.
We have a situation where the measure of a mind in a substrate seems to mean something. With entities getting multiplied like this we are being pushed in the direction of many-worlds cosmologies. We need to think about what it means to say that minds, or any other objects, "exist." This is a matter for the next article.
Conclusion
A thought experiment has shown that the substrate on which a mind exists is not completely irrelevant. When you could be in any one of a number of different situations, on different substrates, the nature of each substrate influences the probability of you being on it. While the substrate may not have a qualitative effect on whether or not a mind can be supported by it, it does, therefore, have a statistical effect when considering substrates that could be supporting you. It follows that this does not just apply to you, but also to others and that an entity existing on a substrate needs regarding, somehow, as multiple entities existing on that substrate, the number of entities, or "measure," depending on the nature of the substrate.
This has possible ethical implications as it could influence the relative value that we place on systems supporting thinking entities. It could also have implications for our future expectations in various situations.
The thought experiment indicates that we need to consider minds and substrates in terms of multiple minds or measure, but our experience of observing thinking entities is that we just see one thinking system: we do not have any experience of this measure other than inferring it from a thought experiment. This apparent paradox will be resolved in the next article in this series, using the many-worlds type view implied by the thought experiment.
The third article will discuss the implications of this. The fourth will discuss the implications for strong AI and Searle's argument, which is invalid, in particular.
References
[1] Searle, J. R. (1997). The Mystery of Consciousness. New York: The New York Review of Books.
[2] Searle, J. R. (1980). Minds, brains and computers. The Behavioral and Brain Sciences 3:417-457.
[3] Web Reference: Strout, J. Mind Uploading Home Page. (2002). Retrieved 22 June 2003 from http://www.ibiblio.org/jstrout/uploading/MUHomePage.html.
[4] Web Reference: Mind Uploading Research Group. (2002). Retrieved 22 June 2003 from http://minduploading.org/.
[5] Bostrom, N. (2003). Are you living in a computer simulation? Philosophical Quarterly, 2003, Vol. 53, No. 211, pp 243-255. (Bostrom circulated a draft of this paper in 2001).
[6] Web Reference: Bostrom, N. (2003). Are you living in a computer simulation? (An online version of the article in reference [5]) Retrieved 8 September 2007 from http://www.simulation-argument.com/simulation.html. (Further information about this subject by Bostrom and others is at http://www.simulation-argument.com.)
[8] Egan, G. (1994). Permutation City. London: Millennium. (Fiction).
[9] Drexler, K.E. (1986). Engines of Creation. New York: Anchor Books.
[10] Drexler, K.E., Peterson, C., Pergamit, G. (2000). Unbounding the Future: the Nanotechnology Revolution. New York: William Morrow.
[11] Drexler, K.E. (1992). Nanosystems. New York: John Wiley and Sons Inc.
[12] The Foresight Institute. (1986-2007). Retrieved June 22, 2003 from http://www.foresight.org/.
[13] Web Reference: Zyvex. (n.d.) Retrieved June 22, 2003 from http://www.zyvex.com.
[14] Reynolds, A. (2000). Revelation Space. London: Gollancz. (Fiction). Chapters 11-12.
sloppy thinking
> Although we appear to have two computers in the box, they are both doing the same processing and we really have
> a single computer with some redundancy, so we will call this new box "Computer AB."
Putting two computers in the same box does not alter anything about them in any sense except cosmetically. From this however the author waves a wand and turns them into a single computer "with some redundancy". This is too sloppy to be of any use.
Either computers A & B are redundant or they aren't. Redundancy I take to mean, the processing is identical - the simulations running on each are in the same state at the same time. ("Some redundancy" is meaningless - if the processing is not identical, they are not redundant because they are separate simulations.) If they are redundant, then it makes no difference whether they're in the same box or not. Is the simulation running on Computer C redundant as well? To make this a precise thought experiment you have to define redundancy and specify which simulations are redundant before you get into the experiment itself. Furthermore, if you have multiple simulations and they are redundant (identical), does it follow that you have multiple consciousnesses? I don't know, but this reminds me of asking how many angels can dance on the head of a pin (I believe the answer is 6). If none of the simulations are redundant, then the author's whole premise vanishes.
Also, there's a big problem in this article with confusing computers with simulations. You could theoretically run thousands of simulations on one computer. From this it becomes clear that the physical computer is not really the substrate. For example, if your simulation models neurons, it's sloppy thinking to identify the computer's hardware as the substrate - that's just a machine that enables the computation. Rather, the substrate is the logic that you create to model the interactions between your neurons and everything else in the simulation. It's a virtual substrate.
Reply to Terren's "Sloppy Thinking" comment.
You seem to think that I see things in terms of some crude "counting of computers", which is strange as the article made it clear that such a position is incoherent, so I would hardly take it.
You point out that changing the casing is purely cosmetic - and I would agree. You refer to me waving a "magic wand" to turn two computers into a single computer with "some redundancy" and say this is "too sloppy". Why? I did not "wave a magic wand". Rather, I did something which cannot have had any profound import at all by any sensible standard, and I said so - hardly an act of magic.
You take issue with the term "some redundancy" as if it implies "partial redundancy" or "partial similarity of computation". It was intended to mean no such thing, but merely to acknowledge that there can be varying degrees of redundancy: you would get more redundancy with 10 computers in the box for example. I would rather not quibble about minor language issues, but since I have to, The Oxford English Dictionary uses the term "high degree of redundancy" - clearly implying that redundancy can occur with varying extent - in an example in the definition of "redundancy", so redundancy is not "all or nothing". If you can have "degrees" of something you can have "some of it". I will accept that the word "some" may be redundant (in the more usual sense of the word) here.
I did not expect automatic acceptance of what you call this "magic wand". I further used the example of moving the two machines so close together that they were effectively merged and impossible even to resolve sensibly into two machines: you did not deal with this, but I see that with a complete rejection of the concept of physical substrate you would hardly see the need.
>Also, there's a big problem in this article with confusing computers
>with simulations. You could theoretically run thousands of simulations
>on one computer.
Again, I would point out that I do not suggest we can simply count computers and that the thought experiment actually shows this to be incoherent. The difference between our positions is that I do not go on to reject the concept of matter substrate.
>From this it becomes clear that the physical computer is not really
>the substrate.
That is what "substrate" is generally taken to mean.
>For example, if your simulation models neurons, it's sloppy thinking
>to identify the computer's hardware as the substrate - that's just a
>machine that enables the computation. Rather, the substrate is the
>logic that you create to model the interactions between your
>neurons and everything else in the simulation. It's a virtual substrate.
We can add as many layers of "virtual substrate" as you want. Of course, a program written in interpreted BASIC could be said to be running on the substrate of the BASIC interpreter, for example, but even if we decide to consider virtual substrates, this does not say why we should not consider the physical substrate. You seem to think it is incoherent to talk of physical substrate. I disagree.
As an example, if all talk of physical substrate is incoherent then any discussion of whether you are in box A, B or C must also be incoherent. You would have to say simply that you are being based on the simulation's logic which happens to be supported by A, B and C and that you exist by virtue of computation occurring in A, B or C. This would tell you nothing about what to expect if something were going to happen to one of these simulations. For example, suppose a hacker were planning to introduce pink unicorns into Computer B, or Computer C, or the box called "AB" (maybe assuming we have merged the contents to some extent) next Wednesday. What is the probability that you will see pink unicorns? I would suggest that without any consideration of which box is causing your experiences (which inevitably involves us in physical considerations of "where") you cannot even make sense of a very simple question. Nor does it help to say that pink unicorns in one simulation would make it different to the others. Yes, I know that. Until it happens they would not be different and the issue of how likely it is that you are going to see pink unicorns is a legitimate question.
I could also find problems with your position even if we allow simulations to be slightly different. For example, suppose there were 10^1000 identical simulations, all the same, of your mind in a virtual reality where nothing abnormal happens tomorrow and 10^1000 identical simulations of your mind in a virtual reality where the Sun explodes tomorrow. I think that your logic would say that, irrespective of any physical details of these computers (you can't consider the physical substrate or admit it as a coherent concept) there is a 50% chance that you will see the Sun exploding tomorrow. Now, suppose someone makes just ONE additional simulation, slightly different to the others, of you in a world where the Sun also explodes, and there is nothing to differentiate it immediately to you from the others. If all we are doing is counting "logic substrates" we would be in a situation where the addition of that single computer to 2x10^1000 other computers changes your probability of seeing the Sun explode to 2/3 as there is one unique way for it not to explode and two unique ways for it to explode. I suggest that is implausible, especially since this extra computer could be made by changing a single bit in one of the 10^1000 computers which are set to do the Sun explosion simulation, and that bit could be flipped by changing a single particle of matter. Expectation, in your view, seems a fickle thing.
You also have the problem that you seem to be relying on counting unique computational states, yet even assigning a computational state to a system of matter actually requires interpretation, something that I will not go into in detail here and now: it will feature in later articles, but as a quick point on this, even extracting the 1s and 0s from the matter in these computers requires interpretation. Who is to say these are the only states? You might say that each computer changes computational state a million times a second yet someone else could use a different interpretation of what the matter is doing and extract, for example, intermediate states between your states, or states extracted in a more complicated way, and say that these are in fact the true states. It is hardly like the 1s and 0s are labelled. Who decides what the computational state of one of these computers is?
Actually, this relates to how things will go later. I will be agreeing with the idea that computation is a reasonable way of looking at it in a later article and discussing how different substrates actually allow us to algorithmically extract different numbers of computational states that are candidates for providers of your experience in the thought experiment. I have never rejected computation, but simply the incoherent idea that substrate is not an issue.
clarifications
Paul,
The wand-waving I'm talking is very explicit in your experiment. The point at which you merge (physically) the Computers A & B to create a 'redundant' computer (in the sense of twice as much matter, used inefficiently), is the magical moment. Why is it magic? Because before the merge, Computer A is running Simulation A and Computer B is running Simulation B. After the merge, the single computer can only be running one simulation. Therefore, you've merged simulations as well. Computer AB is now running Simulation AB. If each simulation identifies with a single consciousness, then it means Consciousness A and B have now merged to become Consciousness AB.
If you accept that, then you have to account for how two consciousnesses can become one. What does that mean? I don't have the answer and I'm not taking a position here either way (again, how many angels can dance on the head of a pin?). But it's an issue I believe you need to address for your thought experiment to be coherent.
If you don't accept that two consciousnesses are merged into one, then you need to explain how you could merge two computers (physically) and yet retain two simulations. Have you forked the process? Is the single computer now doing twice the computation it was before? Is each simulation now running half as fast? This all obviously goes beyond a mere physical merging of the computers.
On to substrates. Strong AI asserts that mind is independent of substrate because it's basically a specific kind of computation, and computers can be implemented in any number of ways, with any number of materials, and here's where I'm making my point - computers can be implemented virtually. Java programs actually run on a virtual computer, as you no doubt are aware.
In the same sense then, a simulated brain is a 'virtual substrate'. Each time a simulation is instantiated, a new substrate is instantiated with it. If you have 10^1000 simulations running, you also have 10^1000 different substrates in virtual existence. If you insist on naming the physical computer as the substrate, then that represents an operation where you're taking the substrate of the substrate. In the example of the human brain, for example, what's to stop us from saying that atoms are the substrate of human consciousness?
In short, when we identify the substrate of consciousness, we must restrict ourselves to only the first abstraction, from which the consciousness emerges, even if the result is virtual.
Further reply on "Sloppy Thinking"
My apologies to readers for posting my previous reply as a new comment, instead of as a reply, as I should have done it.
>The wand-waving I'm talking is very explicit in your experiment. The
>point at which you merge (physically) the Computers A & B to create
>a 'redundant' computer (in the sense of twice as much matter, used
>inefficiently), is the magical moment. Why is it magic? Because before
>the merge, Computer A is running Simulation A and Computer B is
>running Simulation B. After the merge, the single computer can only
>be running one simulation.
Although you have said I am "waving a magic wand" I am not actually making the claim that anything profound happens during the merging, or when I "wave it". You are. I think you are closer to claiming magic.
>After the merge, the single computer
It magically became a "single computer".
>can only be running one
>simulation. Therefore, you've merged simulations as well. Computer
>AB is now running Simulation AB. If each simulation identifies with a
>single consciousness, then it means Consciousness A and B have
>now merged to become Consciousness AB.
You can't prove this. It may seem intuitively obvious that one computer = 1 simulation. Suppose we have Computer A running Simulation A and Computer B running Simulation B and let's say we have them running on mechanical computers with lots of empty space amongst the structure (for the later merging). Suppose we pack cheese around A in all the empty parts. Would it stop simulating? I am sure you do not think so. Suppose we packed bubble wrap or random mechanical components into all these empty spaces? Again - I am sure that you would not think it would not be running a simulation, as long as all this stuff does not jam it up. Yet - when the computer running Simulation A has the computer running Simulation B packed around it you are actually claiming this as a special case and now saying that it cannot be running any simulation on its own terms. Why? Computer A is still there. Whatever it was doing before all the parts to run Simulation B were packed around it, it is still doing. On what basis you can declare Computer A not to be running Simulation A and Computer B not to be running simulation B, merely by having components surrounding them that are doing a specific form of computation?
>After the merge, the single computer can only
>be running one simulation.
which is exactly what I am contesting. You are saying that I am wrong because I differ with that, when in fact that is the very substance of our disagreement. In fact, I would say it is only your semantic preference that this is even a single computer. It could be interpreted in many different ways. Clearly, calling it a single computer is convenient if we want to sell it, repair it program it. It fits into the world of human utility as a single computer.
I am not specifying, at this stage, exactly how to calculate the probabilities. The merging argument is more destructive than anything. It shows that we do have to take account of substrate, but the value of 2/3 for AB and 1/3 for C is not something that I would push as accurate. It results from a fairly naive consideration in which we ignore the possibility of a single simulation being spread out over A and B, for example - something which would actually make substrate more relevant, rather than less relevant, anyway.
>On to substrates. Strong AI asserts that mind is independent of >substrate because it's basically a specific kind of computation, and >computers can be implemented in any number of ways, with any >number of materials, and here's where I'm making my point - >computers can be implemented virtually. Java programs
>actually run on a virtual computer, as you no doubt are aware.
Yes, that fits with strong AI, pretty much how I described it myself I think. We don't disagree about that. Clearly, I am disagreeing with that idea that substrate is totally irrelevant, but I am not asserting that it is relevant in any profound sense, as Searle would, merely a statistical one.
You seem to be mixing definitions of substrate. You have insisted that the substrate is "logical" in your previous post, yet when you say that "mind is independent of substrate" you are clearly referring to a physical substrate when you talk of "different materials".
>If you insist on naming the physical computer as the substrate,
as, arguably, you just did in your definition of strong AI.
>then that represents an operation where you're taking the
>substrate of the substrate. In the example of the human brain, for >example, what's to stop us from saying that atoms are the
>substrate of human consciousness?
Nothing, but I am not maintaining any claim of an absolute bottom level substrate. We could use atoms, subatomic particles, etc. However, any assessments of expectation in thought experiments that we do at a given "level" of substrate should make sense.
One problem you will always have with this sort of view is that if you are saying 1 computer = 1 simulation (if we ignore computers containing obvious parallel processing elements like transputers) you have a big problem formally defining where the edges of a single computer are. Really, a different way of looking at my argument is that it is practically impossible to say where the edges of a computer are and where "another computer starts" in any formal way. All you have is what "looks like a computer".
We will never agree on this. I say that nothing "magical" happens during the merging process. I do accept that something *may* occur, but for it to occur we need to have good grounds for thinking it would occur, beyond just declaring the merging philosophically significant. I say this because there is no obvious physical process that we can point to to do it. You are saying something does happen, even though you can't say how it works or what it is, because it seems obvious to you that it must.
I think I see now
Hi Paul,
It should be obvious now that I'm arguing for the Strong AI position that substrate is (completely) irrelevant, so your claim to the contrary was very interesting to me. You're saying that the substrate has a statistical relevance. I'm going to argue in this reply that your claim is only a reflection of epistemological uncertainty and is thus meaningless. But first, regarding redundancy:
I think I see now what you mean by merging computers. But let me make certain, because I fear I've made too many assumptions already about your position. In order to merge computers in the way you describe in your thought experiment, the computers must be identical in every way. If we're dealing with an electronic computer, the voltage/current characteristics of every wire (as they change through time) must be the same so that if two corresponding wires did come into contact, neither wire's voltage/current characteristics would be altered. If it's a mechanical computer, the position and momentum of every part (relative to all the other parts) would be identical, so that when corresponding parts come into contact, no momentum is imparted to either 'part' as they merge.
If this is what you had in mind, then please excuse my thickness, and yes, I agree that you can partially merge computers in such a way that it is impossible to state with any certainty whether you have 1 computer, 2 computers, or something in between. Likewise for the simulations and consciousnesses running on them. (May I suggest, if this is indeed what you had in mind, that you make this more explicit in your description of the thought experiment - it was not obvious to me).
This uncertainty however is a problem, in regards to your attempt to make a claim about substrate relevance. It derives not from any absolute or ontological sense of what exists and what doesn't. Rather, it derives only from epistemological blurriness about our models of what is a computer or a simulation. Computer AB might be one computer and it might be two. It is whatever we say it is. Either way, nothing about the actual scenario changes.
A good illustration of the problem is to realize that if your merging operation is valid, so is its opposite - a splitting operation. One computer can be split into two identical copies, in principle. Each copy can be split again ad infinitum, meaning ultimately that one computer/simulation is equivalent in a logical sense to an infinite number of identical copies of itself. So one simulation equals two equals 1000 equals infinity. Thus, "statistical relevance" is totally meaningless, since it's nonsensical to apply statistics to an entity that can be represented by any number from 1 to infinity.
Finally, a few clarifying words about substrates themselves. I strongly reject any notion that the "primary substrate" for consciousness must be physical. (By primary substrate, I mean the level directly below the phenomenon of interest). Yes, ultimately, there must be a physical substrate for anything we can observe, but what we're interested in as AI/A-Life researchers is how consciousness may emerge directly from the dynamics of a 'lower level' of interaction. In the case of the human brain, the primary substrate is the interaction of neurons, neurotransmitters, ion currents, perhaps glial cells, and so on. If however we insisted for some reason on examining the substrate of the primary substrate, we'd have to focus on how cells self-organize or emerge from constituents (e.g. proteins, DNA, lipids, messenger molecules, ion concentrations, etc.). We don't expect those details of cellular organization to be pertinent to how consciousness arises, because most of the details at the cellular level can be abstracted away, with respect to the interactions of large numbers of cells. We'd be focusing on the wrong level.
Likewise with a simulated brain. We develop a model which is implemented virtually (logically). We attempt to capture the dynamics and interactions of neurons etc. and represent them as logical operations that unfold in the context of a simulation. That virtual model is the "primary substrate". The substrate of the virtual model is the computational context whose execution enables the model to be instantiated and simulated; that is, the program itself that implements the simulation (which is still virtual). We might not get to a physical substrate for a few more layers!
The key point here is that - as with the irrelevance of cellular organization on the emergence of mind from the brain - only the direct substrate of the (simulated) consciousness is relevant, as is the question of whether that primary substrate is physical or virtual. Insisting that a substrate is not relevant unless it's physical ignores potentially many important intermediate levels (especially the primary one!) and is akin to insisting that we won't be able to describe how a living cell organizes itself until we can do so in terms of how quarks combine to form electrons, protons, neutrons, etc.
Reply to Terren's "I think I see now"
I understand that you are arguing for the strong AI position. I do not regard myself as arguing against strong AI, as someone like Searle is. Rather, I think that the strong AI position requires some clarification. The main thing that I think needs clarifying is how symbols are extracted from physical systems – an issue which relates to what we are discussing here. Any apparent problems with combining or splitting machines are really a special case of problems with vagueness about extracting symbols – and therefore, computational states – from physical systems. Some people may think I am actually arguing against strong AI – but I doubt that Searle would approve of what I am saying. In some of my earlier response to your comments I wrongly assumed that you were taking the view that all that matters is whether something runs an algorithm or not, and that not just the substrate, but even the number of computers running it are irrelevant. Some of my answers may therefore have appeared strange.
I note your point about it possibly not being clear what is meant by combination of computers. Your recent description of what is intended in the thought experiment is correct: it is about combining machines of the same design which are in the same computational state; for example, in mechanical computers this means having all the gear cogs, levels, etc in the same positions and with the same velocity vectors. I am writing an extra article, “Part 1.5”, to go before “Part 2” (which has yet to be written) and this will clarify some of this. It will also provide a more detailed example of the combination thought experiment with a specific computer architecture, which I will call “FlatMech”, based on mechanical computation. I will use some of the text from a draft of this article in this reply to save time.
You say “Computer AB might be one computer and it might be two. It is whatever we say it is. Either way, nothing about the actual scenario changes.” I actually agree with this – in fact it is central to the problem. This is one example of the whole problem of the vagueness about extraction of symbols and computational states from physical systems in the typical strong AI position. I do not think that this precludes making reasonable judgements about measure, however.
Let us consider a simple thought experiment:
Suppose we had 1,000,000 computers running simulations of you in a virtual reality, maybe the same or maybe not all the same – but all of them are possible candidates for your current situation, based on what you know – and that all of these simulations involve pink unicorns appearing tomorrow. Suppose also that we have one computer running a simulation of you in a virtual reality in which pink unicorns do not appear. What probability would you think you would have of seeing pink unicorns tomorrow? Hypothetical situations like this force us into considering what is going on and just where we are getting out distribution of possible situations. When we start to think about computers in various stages of combination it becomes clear that any simple rule of saying “This is one computer. That is two computers” breaks down – yet we must get our probabilities somewhere. At this point we are being forced into considering substrate nature. Ironically, if I produce a situation with two computers in some degree of combination and you say that they are not combined enough to be two computers, or that they are combined enough that we don’t know then, in a way, you are already accepting the very thing that you want to avoid – you are admitting consideration of substrate into your reasoning. In fact, I would argue that even counting computers in the sort of situation that we described is effectively considering substrate because we are not just considering the computation, but what is physically going on into reality to get it done. The problem of substrate nature is very obvious and, even if some argument like the one you made about “infinite minds” were successful I do not think that it would save complete substrate dependence: it would just leave us with a view of statistical dependence that does not seem to make sense, given considerations of infinity, and a view of no statistical substrate dependence that does not seem to make sense given the incoherence of referring to separate machines. We would just have a mess which is open to anyone resolving the problems in either position, but strong AI in the form that many people know it would not be secure.
I think that the main point in your comment was that the conceptual possibility of unlimited splitting suggests that each computer contains an infinite number of minds, and that we can therefore not do any comparison based on numbers of minds. Here is an answer to it:
One response to this objection would be to say that there is a limit to splitting as eventually we will encounter fundamental particles of matter or the uncertainty principle will make further splitting meaningless. This would be a weak reply. We cannot be sure that there are indivisible particles and, even if there are, it would be strange for a view of what minds are to rely on this sort of specific physics: it would imply that minds could not exist in a universe without indivisible particles. The same goes for the uncertainty principle. Furthermore, these kinds of reply could be countered by saying that the splitting is conceptual, not physical, and is therefore not limited by physics. I need a better reply than this.
I may seem to be proposing that a computer has a single mind and that when two computers are combined we have two minds, but I am not. I stated in the article that the idea of “redundancy” or “inefficient use of matter” as a way of determining measure or probability is merely a placeholder – a simplification of the true situation.
I may also seem to be proposing that the measure or probability actually comes from the combination of computers in some sense, as if this is all about combination of machines. I am not proposing this. The article has not stated where we get measure or probability, but merely shown that substrate must affect them and that, all else being equal, minds in high redundancy systems have high measure. It should be noted that I used the word “measure” a lot: I did this to avoid too much discussion of “numbers of minds”. The argument does not really explain what “measure” is, beyond clearly implying that it relates somehow to “numbers of minds”, but this may be in a very abstract way. The argument makes it clear that the sort of probability in the thought experiment is supposed to be greater when we combine computers and we cannot make probability abstract: it has to represent our actual expectations. We seem to have a situation, then, where we may have issues when we try to talk about absolute numbers of minds, where we have to use some kind of abstract “measure” of minds instead, but where this abstract measure is supposed to imply real probabilities. There is no problem with probability splitting indefinitely: a probability of 0.5 can become 0.25, then 0.125, and so on without any point at which this must stop, provided that probabilities are based on some abstract “measure” rather than absolute numbers of minds. Probabilities are not based on any absolute measure, but on relative measure and this is all that we need to be able to deal with. As an example, if there is a probability of 0.2 that your experiences are based on Computer A and a probability of 0.1 that they are based on Computer B then the “measure” of minds like yours in Computer A is twice what it is in Computer B. We could take this as meaning that some absolute number of minds is involved, but as long as we just need to make comparisons of measure we need never confront it.
Combination of computers is not an important process in itself, beyond its usefulness in demonstrating that there must be at least statistical substrate dependence. An argument about infinity may seem strong, but it will not make the problem of combination go away. Given that the thought experiment in the article showed that “counting computers” as a way of assigning probability is incoherent, if you want to assign probability to different situations, which you have to be able to do to make sense of reality and put any philosophy to any use, you need to deal with the implications of counting computers.
Infinity itself need not be a problem, though how we get “measure” is relevant. Some people may still be very sceptical of the idea that infinity can be dealt with by basing probability on “measure” of minds rather than absolute numbers. Situations like this do not necessarily prevent us from extracting useful information, however. An example is the derivation of the basic method of differential calculus. We define the gradient at a point on a curve as the vertical step divided by the horizontal step over some small section of the curve and then we let the size of this section tend to zero – as if it were becoming infinitesimally small. The objection could be made that we cannot compare infinitesimally small values, but we never do compare them. The size of the section tends to zero, but does not reach zero, and we can make a useful comparison of the vertical and horizontal steps. This is not a comparison for a zero size section of curve, but nor is it a comparison for any particular absolute size: it is something more abstract. We do not need absolute values to make a comparison.
It is the same with the issue of “infinite minds”. Even if we find ourselves having to infer infinite amounts of minds, it is relative numbers that we are interested in for the purposes of assigning probability and value. We can do this without stating absolute numbers of minds by using some method which considers some part of the infinite distribution of minds associated with one or more substrate and considers relative numbers of minds for assignment of probability and value as the extent of this distribution tends to infinity.
It may appear that I am trying to appeal to some vague process to save the argument, but all this is actually going into Part 2 of this series. I should, however, at least give an idea of how the distribution of minds will be deal with:
The method that will be later described will be based on the number of ways of algorithmically extracting a computational state corresponding to a mind from a particular substrate. One problem with strong AI is that what is physically going on needs interpreting to produce symbols and computational states and there is no reason why any interpretation cannot be imagined. I will formalize these interpretations by saying that each interpretation of a physical system corresponds to an interpretative algorithm that produces a mathematical structure. Next, rather than being accused of arbitrarily selecting “obvious” interpretations (a charge which Searle makes against strong AI) we implicitly accept all of them. We therefore have an infinite set of interpretive algorithms which can be applied to reality to produce various computational states. Of course, not all of these computational states would correspond to anything like minds, or anything like your mind. In the case of two computers, A and B, we would be interested in those algorithms which can be applied to those parts of reality that would be considered part of Computer A and those parts of reality that would be considered part of Computer B. We seem to have the “infinite minds” problem here. We can resolve this by saying that any interpretative algorithm has a length L and that probability and value are based on comparisons made between numbers of computational states extracted from the reality “in the vicinity” of Computers A and B for interpretative algorithms with maximum length L. Then we let L tend to infinity and we can still make a valid comparison. This is really not much different from the standard strong AI position. It only differs in that it attempts to formalize the stage where we extract symbols, meanings and a computational state from a physical system. The strong AI position typically ignores this.
Trying to get back to some simplicity here, we can link this to what was said in Part 1 about “inefficient use of matter” leading to more greater measure and probability. This is not a fundamental rule, and it may be hard to formalize, but it seems that this sort of thing should follow from the article’s thought experiment. The explanation for this is that the sorts of computers which have “high redundancy” or are making “inefficient use of matter” are those which to which more interpretative algorithms can be applied to produce computational states corresponding to minds – and, of course, when we say “more” here we are simplifying, given what was just said about infinity. Computers that use more matter than needed will tend to provide “more stuff” from which interpretative algorithms can extract computational states.
This will be dealt with in more detail in Part 2, and some of this has been discussed here in advance of me writing it up properly. For now, I am merely suggesting that measure and probability tend to increase when you combine computers and decrease when you split computers, the way in which this measure is computed needing deeper consideration.
substrate still irrelevant
Paul,
To sum up the exchange to this point: you put forward a thought experiment in which you claim to demonstrate that the physical substrate underlying some number of simulated consciousnesses was *not* irrelevant, in the sense that the number of identical simulations running on it had a statistical impact on the probability of a hypothetical consciousness 'ending up' in one of those simulations. The probability is related in some way, presumably, to the overall redundancy of the simulations, i.e. how many identical copies of them are running.
I eventually produced a coherent objection by demonstrating that you could, in principle, split such a computer/simulation into identical copies of itself, ad infinitum, such that any one simulation is logically equivalent to an infinite number of copies of itself, rendering the concept (and math) of "statistical relevance" obsolete.
In your latest reply you attempted to show that by positing a possible infinity of minds via the splitting operation I defined, I must necessarily contend with the details of a specific physical substrate. Thus, substrate is not completely irrelevant in my attempt to show that it is. Also, you asserted that we can say something meaningful about probabilities in your thought experiment even if we can't rely on concrete numbers of running simulations. Instead, you invoked an 'abstract measure' of mind that is relevant in a relativistic sense. Scenario A is half as probable as Scenario B, in other words, even if we can't be sure of the actual probabilities.
To address your objection to my splitting operation, I'm going to show in this reply that we can imagine a scenario where you still have to deal with a potential infinity of minds in a way that is completely independent of the substrate. In doing so I will show that substituting an "abstract measure of mind" for hard numbers doesn't solve anything for you.
I'm going to propose a variation on your thought experiment, but I need to lay a little groundwork first. To that end: I argue for Strong AI in the simple sense that if you replaced every neuron in your brain with a synthetic equivalent, you'd still be conscious, and in fact you'd have absolutely no idea that anything were different. That's the most simple assertion, I believe, from which Strong AI proponents may conclude that substrate is irrelevant.
To develop this a bit further, if you replace every neuron in your brain with a computer chip, you wind up with a synthetic brain. For clarity, let's state that it's not a computer in any conventional sense at all. For instance, it's not running software. It'd be more accurate to say you've got billions of simple computers running in tandem. But instead of calling it that, let's just refer to it as a synthetic brain.
Now, going further, consider that it's straightforward to model each of these synthetic neurons in software, and also to model the interaction of all these synthetic neurons, and the context or environment in which they function. Now we take this (enormous) model and run it on a computer. This is now a computer running a simulation of a brain.
Strong AI says that both the synthetic brain and the simulation of the synthetic brain are conscious, and not only that, but if you replaced my brain with either, I'd never know the difference, assuming the sensory inputs to each were identical. (As an aside, the real mindblower here is that you could start and stop the simulation, save it to disk and restart it later, and my conscious experience would remain uninterrupted, assuming the inputs stopped and started with the simulation.)
Now we're ready for the thought experiment. Let's start with a living brain, sitting in a lab, being kept alive by advanced technology. Every conceivable input to the brain is attached to simulated data feeds, such that the consciousness associated with the brain is experiencing sitting on a beach with ocean waves tumbling, smelling that wonderful salty air and feeling a warm sun beaming down on his/her internally-embodied face. Sitting next to the brain is its synthetic equivalent, in which each neuron has been replaced with a tiny chip. It too is getting the same data feeds as the living brain. Sitting next to that is a computer, which is running a simulation of the synthetic brain sitting next to it. The simulation is fast enough to run at the same speed as the brain and the synthetic brain, and it too receives the same data feeds as the other two 'brains'. Now, to make this even weirder, we have a second computer sitting next to the first computer. The second computer is running a program that simulates the physical internals of the first computer (similar to, for example, an arcade game emulator you can run on your PC). The second computer's instance of the first computer is also running the same simulation as the first computer and getting the same data feeds. Then we add a third computer, which simulates the second computer's internals, and so on. We could go on for infinity here. It's important to note that at each step of the way, we might be using completely different kinds of computers, electronic, mechanical, made up of unique materials, designed in unique ways.
As bizarre as it sounds, each of these computers is ultimately supporting a simulation of consciousness that is identical to the consciousness associated with the living brain. And just as bizarrely, we've got another splitting operation - another way in which, given a single simulation of consciousness, we can conceive of an infinite number of copies of that simulation, without any dependence whatsoever on a specific kind of substrate.
This result shows why invoking an 'abstract measure' of mind, in place of counting computers, doesn't help. The above splitting operation shows that you can't escape the uncountability of identical consciousnesses. Infinity isn't the real problem here, although it's a symptom. The real problem is that it's totally up to you to define, in your thought experiment, whether you have one million identical consciousnesses, just one, or an infinite number of them. You could never argue that such a number (or any kind of measure, abstract or not) was correct or even incorrect, because I could just as easily pick another number (or measure), and I would be no less correct (or incorrect).
Reply to Terren's "substrate still irrelevant" comment
Thank you for your comments. As before, I will use some text here that I am also putting into Part 1.5: there is just too much to say here to write it twice.
You said: “In your latest reply you attempted to show that by positing a possible infinity of minds via the splitting operation I defined, I must necessarily contend with the details of a specific physical substrate. Thus, substrate is not completely irrelevant in my attempt to show that it is.”
Yes, however I want to be clear that I am not making this up just to deal with these objections. The article states that a more sophisticated view based on extraction of computational states by algorithms will be discussed in later articles, and I have already discussed this issue of infinity in situations like this in previous articles (such as the ones on Occam’s razor), so this is how things have been intended all along.
You said: “Instead, you invoked an 'abstract measure' of mind that is relevant in a relativistic sense. Scenario A is half as probable as Scenario B, in other words, even if we can't be sure of the actual probabilities.”
The issue of infinities may mean that measure is only meaningful in a relative sense. As probability is based on measure this may seem to indicate that probability is only meaningful in a relative way. This would not be entirely true. If there a number of situations in which you could be, and you find the measure for you being in each, relative to the total measure of all these situations, then you know the probability of you being in each, relative to the total probability of all these situations. This may seem not to be telling you much about anything beyond these situations. If, however, the possible situations being considered comprise all possible situations then the probabilities obtained from the relative measures of these situations are absolute probabilities.
For example:
Suppose your experiences could be due to one of two possible substrates, A and B, and:
Measure for you being on A = 0.8 x Total Measure for A and B
Measure for you being on B = 0.2 x Total Measure for A and B
then we can say that:
Probability of you being on A = 0.8 x Total Probability for A and B
Probability of you being on B = 0.2 x Total Probability for A and B
If you know that you could be in no other situation other than being on substrate A or B then:
Total Probability for A and B = 1
and the probabilities now become absolute:
Probability of you being on A = 0.8
Probability of you being on B = 0.2
You might assign absolute probabilities if are sure that you the situations being considered comprise all possible situations, or as an approximation if you assign other situations negligible probability. It is not necessary for all situations to be well defined, or to involve just a single substrate. For example, one situation being considered might involve any substrate of a given general type, or may comprise any situation which is not part of the other situations, so that all possible situations are included by definition, but assigning measure and probability might need a lot of assumptions. Assigning absolute probabilities does not mean that you are declaring complete knowledge about the nature of reality. You might accept the possibility that the possible situations being considered are “contained” in one of a number of other possible situations about which you know little, but nevertheless are an accurate description of the possibilities for practical purposes.
This issue of relative and absolute probability is nothing special about what we are doing here: it is an issue for probability calculations in general. It might be more apparent here because of the wide scope of what is being considered.
You said: “The real problem is that it's totally up to you to define, in your thought experiment, whether you have one million identical consciousnesses, just one, or an infinite number of them. You could never argue that such a number (or any kind of measure, abstract or not) was correct or even incorrect, because I could just as easily pick another number (or measure), and I would be no less correct (or incorrect).”
Well, the idea of any particular number being relevant is something I will be rejecting anyway. The main point of this objection seems to be that the way in which we split computers is arbitrary. I could choose one way of splitting a computer into many computers and show that two substrates have a particular relative measure, and then I might be able to use another method which gives significantly different relative measure.
The sort of splitting used in the thought experiment in Part 1 is not supposed to be a formal method for determining measure, but is rather part of an argument to show that some concept of measure is needed and that, all else being equal, high redundancy, or inefficient use of matter in computation, is a likely indicator of high measure. The objection, however, ignores this and tries to treat it as a formal method for determining measure. Even with an informal method such as this, however, multiple types of splitting can still be coherently discussed as follows:
Suppose we have a computer running some entity, E. We define a splitting operation, S1, which splits the computer into different computers running E. We define an index, I1 such that each computer obtained by the splitting operation S1 is at some point on this index. For now, let us assume that we are dealing with finite numbers of computers resulting from the splitting operation.
The measure of E after the splitting operation is the number of computers on index I1.
Now, we define a new splitting operation, S2, which splits a computer running E into other computers running E, and an index, I2, such that each computer resulting from splitting operation S2 is at some point on index I2. We will presume that we are dealing with finite numbers of computers for this index too.
Suppose we apply splitting operation S1 to some computer running E and obtain a distribution of computers along index I1. Now, we want to apply S2, but we have already done one split. We deal with this by applying S2 to each of the computers on index I1, generating an index of computers I2 with a number of computers on it in each case. We now have index I1 containing many versions of index I2 arranged along it, each containing a number of computers. We obtain the total measure of E by counting the computers on each index I2 along I1 and adding them all up.
We could do this the other way round. If we had done split S2 first we could then use S1 to split each computer produced by S2.
If we allow splitting to generate an infinite set of computers this does not substantially change the situation. Indexes I1 and I2 would be infinitely long. We could now no longer obtain an absolute measure value. We could still, however, determine the relative measure of E in two computers C1 and C2 as follows:
Let L1 be the maximum length of index S1 that will be considered.
Let L2 be the maximum length of index S2 that will be considered.
Apply S1 to C1, generating computers distributed along I1, only considering index I1 up to length L1. Apply S2 to each computer on I1, generating a different distribution of computers along I2 in each case, only considering index I2 up to length L1. Count the total number of computers along each index I2 along I1 and add the totals up to obtain a measure, M1, for E in C1. This value should not be regarded as meaning anything as an absolute measure.
Apply a similar process to C2, obtaining a measure, M2, for E in C2.
Total measure of E = M1+M2
Measure of E in C1 as a proportion of total measure = M1/(M1+M2)
Repeat all of the above process increasing L1 and L2 each time. Use the value for measure of E in C1 as a proportion of total measure which is converged on as L1 and L2 tend to, but do not reach, infinity.
In this way, relative measure for two computers C1 and C2 can be obtained for two methods of splitting. By extension, relative measure for two computers C1 and C2 could be obtained for any number of methods of splitting.
Further, a single index, I3, could be defined containing all the computers that occurred on each version of index I2 along I1. A single splitting operation, S3, could be defined that combined S1 and S2 to produce I3.
The relative measures obtained from a single splitting operation cannot be considered a final result because there is always the possibility that a subsequent splitting operation could change the relative measure, nevertheless it can be considered to be an indication of likely relative measure after subsequent splitting operations, all else being equal.
It could be argued that we might obtain different results if we applied splitting operation S2 first, followed by S1. Thought experiments involving splitting like this, however, are not formal methods for determining measure, but merely demonstrate statistical substrate dependence, the need for some concept of measure of minds and the general association between measure and redundancy. We should not expect processes like this to give accurate measures or probabilities, and so should not be unduly concerned if we got different results from applying different processes – such as when we do splitting in a different order. The main point is that even with an informal idea of splitting and measure we can coherently discuss application of multiple splitting operations to a computer. This might be answered by pointing out that S1 might be such that the computers that it produces are so different in structure from the original computer that was split that, although S2 can be applied to the original computer, S2 cannot be applied to the computers produced by S1. At this stage the objection is pushing the simple idea of splitting beyond its limits and the response to this has to be:
The simple splitting concept as described in Part 1 is not a formal description of measure but merely part of a thought experiment to demonstrate the need to consider measure and its relation to general “redundancy” in a substrate. A formal idea of how measure results would deal with issues like this.
The issues of possibly getting different results depending on the order in which we perform different splitting operations, and of splitting operations preventing subsequent splitting operations from being applied, arise because of a simplification in these sorts of considerations of splitting – that different ways of splitting a computer cannot use any of the same matter. If this assumption is removed then any “splitting operation” can simply become an operation to “extract” different computers and is unaffected by previous splitting. The objection suggests that there is some arbitrariness in the selection of the splitting operation to be used, but if we are not limited by different splitting operations having to use separate matter then we do not have to select a specific splitting operation: all splitting operations can be applied. A more formal splitting idea that would deal would these issues would therefore have these features:
1. Re-use of matter by different splitting operations: a splitting operation can be applied irrespective of what other splitting operations are applied to the same matter.
2. All conceivable splitting operations are considered as contributing to the set of computers resulting from the splitting.
The proper, formal idea of measure that will be discussed in Part 2 works in this way. Instead of splitting a computer into separate computers, algorithms are used to extract computational states, a more sophisticated idea than extracting computers. Each algorithm extracts a single computational state. Overlap between different extraction algorithms is ignored and any algorithm which extracts a computational state can be applied without regard for other algorithms. There is no need to try to devise different types of splitting operation as the measure is based on the entire set of logically conceivable extraction algorithms (equivalent to “splitting operations”) and any argument about a particular method of splitting being arbitrary, and the choice of splitting method being subjective, is irrelevant.
I am not sure, but part of your objection seems to imply that I have omitted “virtual machines, however there is nothing special about them and these would be included in any formal idea of measure like this. An algorithm which extracts a computational state from a physical substrate could, for example, extract a description of a high level language interpreter, together with the program that it is interpreting, from the physical substrate, and then do further computation to extract a computational state of a mind from this. Whether a machine is virtual or not is more to do with us interpreting what it is doing in a particular way and introducing some abstraction into our view of its functioning.
You seem to have issues with counting things and I am still not sure how you would assign probabilities. I would still be interested in knowing how you would deal with a situation like this:
There are 10^1000 identical computer simulations (i.e. the machines are running the same software), on separate machines, in which a mind like yours exists in a virtual reality, and pink unicorns will appear tomorrow. There is one extra simulation, slightly different to the others, of a mind like yours in a virtual reality, and pink unicorns will not appear tomorrow. You know that your experiences are due to this computation, but either the pink unicorns appearing or the pink unicorns not appearing simulation are both compatible with your current experiences. What probability would you assign of seeing pink unicorns tomorrow?
To put it another way, the question is asking: if you don’t like my method of counting, what do you want to count, or maybe you don’t want to count anything at all? If you don’t want to count anything at all then you can’t assign probabilities.
probability is meaningless
Hi Paul,
I will await your next installment before I comment on your ideas about computational states, particularly if as you say that's the real thrust of your articles in the first place.
And rather than go around in circles I will just attempt to answer your last question, and hopefully that will clarify where I stand with regard to computing probabilities in your thought experiment.
In your scenario, you posit 10^1000 identical simulations in which at a given time in the future, a pink unicorn will appear. Then you take one of them and change it to prevent the unicorn from appearing. What is the probability then that you will see a pink unicorn?
As I see it, there are two ways to answer this:
1. It's a meaningless question
2. I can answer it but I have to make an assumption, and that assumption is arbitrary and thus meaningless
To go a bit further with (1): It's a meaningless question because there's no reason to believe, in light of all the talk of splitting computers and infinities and uncertainty, that we can count or otherwise work with any kind of 'measure of mind', absolute or relative. To get into the 'math', it makes the same amount of sense to say the probability is 1/10^1000 as it does to say 1/2, because I can in principle split the non-unicorn simulation into 10^1000 copies of itself.
For (2). On the other hand, I can talk about probabilities if I make the following assumption: That because a simulation is logically identical with an infinite number of identical copies of itself, then I assume that any number of copies of a simulation is really only a single simulation. So the salient data becomes, how many different simulations are running? In this question, there are two simulations. The fact that there are 10^1000 of one and only 1 of the other is immaterial. Thus, the probability of not seeing a unicorn is 1/2.
But even with the answer in (2), ultimately it's meaningless, because the number I came up with doesn't follow from anything except the way I've chosen to define my terms. You might choose to define your terms differently, or not to define them at all, in which case you won't agree with my numbers. I won't argue with you if you come up with a different number, but I will continue to assert its meaninglessness - once again, how many angels can dance on the head of a pin?
So really, if forced to choose, I'd pick the first answer. For me to reconsider my position on that you'd have to give me some reason to believe that there could ever be a real world situation that might depend on such a probability, since that would suggest that it's not meaningless after all.
Reply to Terren's "probability is meaningless" comment
If I understand it correctly, your position seems to be that you don't really see the need to count anything, and don't consider probability very important (at least in this context), but if you did need to count anything you would count unique algorithms rather than computers. I will answer this now.
Firstly, I had said that Part 1.5 would give extra support to the first article and that Part 2 would define measure properly. I have now decided not to use strange numbering just to try to follow what I said in Part 1. The next article is Part 2, which has just been posted on my own website and at machineslikeus.com at http://www.machineslikeus.com/cms/extra-information-about-substrate-depe.... This article is intended to give extra support to Part 1. The article to follow Part 2, Part 3, will not get round to defining measure properly either: instead, it will deal with the issue of "arbitrariness of explanation".
To deal with the suggestion that this is irrelevant:
It is not about me preferring a particular way of determining probability. Rather, the argument I gave shows that the most obvious method of determining probability – counting computers – is incoherent. There is nothing arbitrary in selecting a coherent view over an incoherent one. Further, I would quote this from my latest article (Part 2):
"The argument may seem irrelevant, and it may seem tempting to write it off as trying to answer a pointless question, but the question is not pointless. Should there be uncertainty about your current status, and if a number of situations involving different substrates are possible, considerations like this have an effect on the very real probabilities of being in different situations. A well-defined position on the relationship between minds and matter is also needed to deal adequately with Searle’s arguments."
I would also quote this from my latest article:
"We do not find it generally tenable to discard probability. A scientific theory may not allow us to easily obtain probabilities of events happening, but one that did not even in principle contain any way of coherently dealing with probabilities would be regarded as useless.
I also maintain that most people who declare probability irrelevant would not consistently subscribe to their own position in all situations. In some situations their expectation of various events happening or not happening would be very important to them. As an example, suppose that you know that your brain has been copied and that you are running on one of a number of computers. On one group of computers you face pleasant experiences and on the other you face horrible torture. The differences between these groups of computers, and which of these differences matter, would be important issues to almost anyone, even though this may not mean agreement with me. A person who declared probabilities to be irrelevant or undefined would find him/herself in an unusual situation: one in which he/she was unsure whether or not to be happy or terrified. This is absurd. What is such a person supposed to do? Is he/she supposed to be neutral about the whole thing? Deciding whether to be scared or not is quite basic."
i.e. probabilities of what is going to happen are quite basic to how we feel.
I may be wrong about how we should determine probabilities, but I think to suggest that it is some kind of subjective preference makes no sense at all. If I am wrong, it should be possible to show that I am wrong. If I am right, I should be able to show that I am right. Probability is a really basic idea and no philosophical view can work while leaving it in the clouds.
You indicated that if you did have to count something it would be unique algorithms (The numbers you gave could only have that as a justification.). Now, the argument I previously made does not deal with this very well. It concentrates mainly on incoherency in the view of counting computers. To deal with the counting of algorithms I would refer to my latest article (Part 2), which is available here at machineslikeus.com at http://www.machineslikeus.com/cms/extra-information-about-substrate-depe.... This gives an answer to this sort of view and shows it to be equally incoherent. One very serious weakness in this position is that of "arbitrariness of interpretation" (my term) or "multiple realizability" (Searle's term). Now, just because I recognize a valid point made by Searle, this does not mean I have bought into his general views on AI, which I don't think make sense at all. However, dealing with the issue of arbitrariness of interpretation without invoking some kind of measure is going to be difficult. The problem is that nothing intrinsic to the physics produces the "algorithms" that you are supposed to be counting. Some sort of interpretation is needed to say that a particular algorithm is running in a particular system. The problem is, in the absence of any specification of what a valid interpretation is (and a philosophical justification of this specification), any algorithm could be "found" in any physical system by using the right sort of interpretation. The argument here is too long to make it worth giving here, and this issue will be getting an entire article next, but the latest article will give an idea of the problem. It is under the heading "The Problem of Interpretation".
You say that you would need a "real world example". I would quote this answer to such an objection from my current article:
None of the situations considered break known laws of physics. It would therefore be unreasonable not to consider them as “real-world” situations unless a very specific definition of “real-world” is used which only relates to things in our everyday experience. If appropriate technology to deal with computers and brains were available these situations would be practical ones. Any discussion of strong AI is considering hypothetical situations involving futuristic technology by definition, and is considering intelligence on substrates other than human brains, so it is inconsistent to suggest that philosophical arguments about it should not enter the realms of futuristic technology or extremely different situations from what we know.
"It is not generally a good objection to an argument to suggest that it deals with situations that are too far detached from everyday life to be meaningful. Whatever view we have of the relationship between matter and minds, it should be valid in all situations. If we can produce situations where we get nonsense results, even if those situations seem contrived and practically implausible, those situations will not go away by hoping that they do not appear in the “real world”. The mere fact that they occur at all would indicate that we had a flawed view that would not work under normal conditions either."
As a last comment, I would argue that there is uncertainty in almost any position we can imagine and that if we really declare probability to be meaningless there is no point in saying anything.
diving into probability
Paul,
I haven't yet read your Part 2 article but just wanted to clarify quickly that I'm saying probability is meaningless in the very specific context of your thought experiment. Obviously probability is an important aspect of any model that undergoes transitions to multiple possible states. However, in your thought experiment, it is too difficult to understand what "transition" even means. Let me be specific.
In your thought experiment you set up three identical simulations of my brain. I find myself in one of those simulations, without being told which. One question is, what does it mean to "find myself in one of the simulations"? This is very strange to think about.
We're wondering what is the probability of my consciousness being associated with a given simulation of my brain. Probability in this context refers to the likelihood of the system to enter a specified state among multiple possible states. However, for us to be able to work with this idea, we need to define a way for the system to transition from the initial state to the state of interest. Specifically: the initial state is the moment before the simulations go online, and the state of interest refers to "finding myself" in a given simulation. How does this transition occur? Can we say anything meaningful about how my organic consciousness might transition to a "synthetic" one?
Maybe a better way to illustrate the difficulty is to recognize that my organic consciousness is still present after the simulations go online. One perfectly valid answer to the question is, there's 0% chance my consciousness will be in one of the simulations. It's right here in my brain, and nothing will change that, no matter how many simulations you run.
If instead you say that the probability you're interested in has nothing to do with my organic consciousness transitioning to a synthetic one, then probability is meaningless, unless you can specify it in terms of how the system transitions from one state to another. In other words, in what sense might my consciousness be associated with one simulation but not another? More importantly, how might that state arise, in opposition to the other possibilities? Unless you can answer that, I can't see how probability is meaningful.
Terren
Get the probability straight first
Don't think the experiment described has anything special to do with brains or consciousness. We just have to make up our mind what we mean by probability. Let's replace the 3 simulations with 3 boxes, and the "transfer of consciousness" with simply throwing a marble in a box chosen at random with equal probability.
All probabilistic considerations in the article still apply to the "box" experiment, including the possibility (with a little mechanical ingenuity) to gradually bring two of the boxes together until they are one (with or without thicker walls, as we please).
This exposes the true problem with the original experiment: we have to get the probability straight first. In particular, the "mysterious" change of probability from 1/3 (before merging) to 1/2 (after merging) has nothing to do with the reliability of the substrate, but with the fact that we don't know *which* boxes are merged. The article says "A and B", but the marble may already be in A or in B.
Did the author mean to say "the boxes *which do not contain the marble* are merged"? Then conditional probability must be used (as someone tried to explain), since we've already ruled out one box for merging.
Did the author mean "two boxes *at random* are merged"? Then we must accept that one of the boxes merged may be the one with the marble.
Either way, I don't see how all this tells us anything interesting about consciousness.