@approxinfinity
I do not mean to understate the risks of technology that we face moving forward.
We are in real danger from quantum clock time increments combined with quantum computing.
We are facing digital runaway, even if computers never become sentient, or simply pass the Turing Test by our being unable to tell the difference between their appearance of sentience and there actual sentience.
The real problem we face is that with quantum clocks breaking time into increments smaller than we can comprehend, except in crude terms of orders of magnitude, quantum computers can begin to make rules that operate near the limits of nano infinities. In turn, whether digital systems become sentient, or remain dumb near the speed of light, initial conditions of digital choice will unleash butterly effects down stream elaborated into a complexity we cannot comprehend, or anticipate. This is not only possible possible, but apparently happening all the time in bit torrent storms we cannot forecast or explain.
We already know we don't know what we don't know about the digital realm that we increasingly float upon like analog bugs on a roiling sea of digital complexity.
Frankly, at this point, it is not really very significant whether AI becomes sentient or not; that is the least of our concerns. We are already far beyond being able to verify the existence of sentient AI even if it were to already exist. Unknowable, unfathomable, complexity near nano infinity is the new normal IMHO.
Thus, humanity's only hope is institutions. Institution must be established that recognize this and so absolutely DO NOT entrust our choices and way of life and moral-ethical standards and epistemoligical framework of knowing things be compromised and subordinated to the inevitable butterfly effects originating near the frontier of nano infinity.
The oceans and humanity's decision to explore them offer a good analog to our situation.
The oceans were to big and too full of complexity at every level for us to understand and anticipate the context they created for us at sea. Those that went to see had over time to both establish rough heuristics about when the sea could and could not be navigated, and establish scientific inquiry to learn and map the shores, the currents and the thermosaline layers, and the weather systems, as we went along. We had to not only map the oceans, but understand that maps could only tell part of the story. A whole new set of institutions--formal and informal--had to be established through experience and thought and experiment. There was no not going to sea. There was simply to much to be gained strategically and economically by going. Similarly, there is no not going into the sea of nano infinity and quantum programming of activities at the nanoscale. There is too much to be gained, both strategically and economically. But sailors over the millennia learned the sea neither to be trifled with nor to be grasped in its entirety. We sailed it, we did not control it. It was a wildly more dynamic context in comparison to the land. Nano infinity is going to prove to be a wildly more dynamic context in comparison to legacy scale of years, months, days, minutes and seconds.
Institutions are the only way to deal with unknowns. Institutions can be fair to many, or unfair to the many. But institutions about how to negotiate the frontier of nano infinity and how to interface the wilderness of nano infinity with the realm of time we now operate in.
It is critical to emphasize that we may have been in digital runaway for quite some time and just don't realize it.
It is naive for any of us to think that we can forecast the point in the future where the sea of nano infinity becomes more than we can handle. It is also naive for any of us to think that we can back cast when digital runaway occurred. All any of us can say is that we better start instituting for it ASAP because it either happened already, or is coming.