Posts Tagged ‘recorded data’

“The Language of Succession”

November 20, 2026

*

The Illusionist.  Ricardo F. Morín.  CGI, 2026.

*

November 18, 2026

Oakland Park, Florida

*

The director of an artificial intelligence company once wrote that human beings would be the first species to design their own descendants.  The curious word in that sentence is “descendants.”  An inventor can describe a machine by telling us what it does.  To call the machine a descendant is to say something further about the relation between the inventor and the invention.  Even if the machine should one day surpass its inventor in every task by which the two can be compared, it would still be reasonable to ask what, precisely, the machine had inherited, and in what sense the inventor could be called its ancestor. [1]

A senator who opposes the direction of these enterprises takes that language seriously.  He has reason to.  A man who directs an artificial intelligence company can commit people and resources to an idea that another man can only entertain.  A proprietor of several such enterprises goes further.  “Propagating super intelligence to the stars is a great success condition for a biological bootloader,” he writes.  A bootloader does its work until the system it starts can take over.  In his analogy, humanity could therefore succeed in the role he assigns it while the question of its own future remained unanswered.  Yet when the senator moves from these remarks to the proposition that machines are already beginning to replace us, something curious happens.  He disputes the desirability of the future these men imagine while granting them considerable authority to say what future their work is bringing about. [2, 3]

Suppose two witnesses have given accounts of the same afternoon.  Each account, read separately, is plausible.  Asked to compare them, a language model observes that if both witnesses have given the correct time, the same person would have had to be in two places at once.  Neither witness made that observation.  The model has drawn a consequence from the relation between the accounts, and a reader can check the inference by returning to the two statements.  Which witness, if either, gave the correct time remains a matter for inquiry.  A more accomplished model might detect discrepancies that escaped a hundred readers.  What such an improvement would establish is how much more an instrument of comparison can bring to human attention.

An international assessment of general-purpose artificial intelligence, prepared by researchers from many countries, describes systems that can answer difficult questions in mathematics yet struggle to recover from elementary errors in longer tasks. [4]  It seems to me entirely possible that reasoning from what has been recorded, finding relations among observations, and suggesting what might be examined next could remain the principal achievement of language models.  A relation that no investigator has noticed may alter the course of an investigation.  Whether a system capable of such discoveries could also come to experience the world, form purposes of its own, or find a way to pursue those purposes against human opposition presents several further questions.  An improvement in inference gives us a reason to investigate those possibilities.  It supplies no measure of the distance between an inference and an intention.

The fortunes committed to the pursuit of superintelligence matter in a quite definite way.  Money can keep an investigation alive through years of failure.  It can buy equipment, attract researchers, and permit a second approach after the first has been exhausted.  Where the obstacle is a shortage of resources, a fortune can change the outcome.  Where the obstacle is that the researchers have yet to identify what would count as success, expenditure cannot settle the question their experiment is meant to answer.  Before an investor can claim progress toward the creation of a successor, someone must be able to explain what observation would distinguish an early successor from a more capable instrument.  The investor’s willingness to pay for either is evidence of the seriousness of his undertaking.  It gives the public little basis for deciding which of the two he is building.

While that question remains open, the same expenditure can produce consequences for which the word “replacement” has a more immediate meaning.  Suppose an employer gives a screening system built on a language model the records of two applicants.  One has held the same position for ten years.  The other left a comparable position to care for a parent and later returned to work.  The employer accepts the system’s ranking and invites only the first applicant to an interview.  The second applicant writes to explain the interruption.  The employee who receives her letter can confirm that the dates were entered correctly.  He has no authority to reconsider what those dates were taken to mean.

For that applicant, the decisive event occurred before she applied.  The employer had already decided whose judgment could affect the outcome and had given the employee who would answer her letter only the power to correct a record.  Her record was accurate.  What she needed was someone entitled to consider whether an interruption in employment bore on her ability to do the work.  Had the employer allowed such a review, the same model, with precisely the same capabilities, could have produced the same ranking and the applicant could still have been heard.  Her loss arose from a decision about the terms on which the employer would hear her.

The senator’s proposed ban on artificial superintelligence makes the distinction consequential.  The bill would prohibit a system capable of planning and carrying out humanity’s destruction or disempowerment.  It would also prohibit a system on the separate ground that its cognitive performance exceeds human capabilities across most domains or tasks.  The prohibition extends to systems readily modified to exhibit either capability and to systems already displaying any of the precursor characteristics specified in the bill. [5]  A legislature may conclude that performance exceeding human capability across most domains creates an unacceptable risk even while a system remains subject to human control.  That conclusion requires an account of how the capability could defeat the controls and why the danger warrants prohibition.  The applicant’s case raises the converse difficulty.  A system of quite ordinary capability may exercise decisive influence over a life because the people who employ it have made its recommendation final.

There is an agreement concealed in the language available to describe what happened to the applicant.  The employer can say that the system rejected her.  A critic can say that a machine has replaced a human judge.  The first description can relieve the employer of the burden of explaining the rejection.  The second can make the employer’s arrangement appear to have followed from an advance in the machine.  Both descriptions leave the applicant writing to the wrong address.  She has no quarrel with the dates in her record.  She has a question for the person who decided that no one would be authorized to hear what those dates left out.

Sources

  1. Sam Altman, The Merge
    https://blog.samaltman.com/the-merge
  2. Elon Musk, statement on digital superintelligence
    https://x.com/elonmusk/status/2106906678293606496
  3. Bernie Sanders, Don’t Let Big Tech Dictate Humanity’s Future
    https://www.sanders.senate.gov/op-eds/dont-let-big-tech-dictate-humanitys-future/
  4. International AI Safety Report 2026, executive summary
    https://internationalaisafetyreport.org/publication/2026-report-executive-summary
  5. Ban Artificial Superintelligence Act of 2026, S. 5493, sections 3 and 9
    https://www.govinfo.gov/content/pkg/BILLS-119s5493is/html/BILLS-119s5493is.htm