banner of LMP department's DP&AI initiative

We are just at the beginning

|
William Hoffman
10 minutes
The tapestry of the human body is coming into sharp relief

Much has been written about patterns in nature. We see them every day.  The science writer Philip Ball titled one of his books about natural patterns The Self-Made Tapestry:  Pattern Formation in Nature illustrating in words and images the remarkable designs and motifs created by simple, local interactions.  

pufferfish skin pattern

The British mathematician, codebreaker, and computing genius Alan Turing after whom the "Turing test" for machine intelligence is named described these patterns as arising from chemical reactions during morphogenesis, the process by which an organism takes shape.  He proposed mechanisms by which spots, stripes, swirls, rows, and ripples -- what have come to be known as "Turing patterns" -- may emerge in living things.

When we think of patterns in the context of a tapestry, we think of woven fabric depicting patterned textile art hanging on a wall.  But tissues can have tapestry-like features as well, a consequence of the self-assembly of biological molecules into organized, patterned sheets, layers, tubes, and structures.  One research group recently observed that cells themselves, in loom-like fashion, “spin and weave” tissues.  These scientists employed a “computer controlled jacquard loom” to weave advanced functional materials from the structural proteins collagen and elastin.  They used weaving algorithms inspired by nature.

The Jacquard loom holds a special place in the history of design and pattern creation but also in the history of automation and computing with echoes in machine learning and artificial intelligence (AI) today.  

Before recapitulating that story, let’s go back nearly half a millennium to the year 1543 -- to two Belgians: a weaver and an anatomist.  The weaver was Willem de Pannemaker whose tapestry workshop was renowned throughout Europe.  A set of his Abraham Tapestries depicting the biblical story of Abraham was delivered to King Henry VIII of England that year.  The anatomist was Andreas Vesalius whose magisterial book published that year, De Humani Corporis Fabrica, often translated as “On the Fabric of the Human Body,” laid the foundation for modern medicine. His “workshop” was a dissection theater, though the term “fabrica” is perhaps better understood as what one scholar termed a workshop where an “imaginative reconstruction” of the human body took place.   On his “History of Information” website, science historian Jeremy Norman observes that Vesalius’s book also broke new ground in the nascent field of printing with “its unprecedented blending of scientific exposition, art and typography.”

The Fabrica was a technological and aesthetic as well as a corporeal tapestry of information.

Weaving looms large in the history of computing and anatomy

Now to return to the Jacquard loom.  Patented in 1804 by Joseph-Marie Jacquard, a French weaver and inventor, the loom used punched cards to control the movement of threads.  That allowed precise, complex patterns to be created.  Each punched card could be imagined as a simple program instructing the loom on how to weave the fabric.  The loom’s woven tapestries were unmatched for intricate detail and exquisite matrices and patterns based on binary code, which was storable.   The programmable computational loom inspired the British mathematician, mechanical engineer and inventor Charles Babbage who conceived of an Analytical Engine.  It also inspired Babbage’s young collaborator.  

Ada Lovelace photo

"We may say most aptly that the Analytical Engine weaves algebraic patterns just as the Jacquard loom weaves flowers and leaves,” wrote Ada Byron Lovelace, daughter of English poet George Gordon Lord Byron and herself a brilliant mathematician.  Lovelace was tutored by the Scottish mathematician Mary Somerville who introduced her to Babbage.

The Babbage-Lovelace collaboration that commenced in the 1830s occurred concurrently with the rise of cell theory in biology.  For more than a century, the prevailing view of tissues, organs, and organisms was that they were composed of fibers.  Fiber was thought to be the minimal building unit of the body.   Life was a fabric of highly organized fibers.  Anatomists at the time used metaphors derived from weaving, textiles and embroidery to describe what they were seeing through the microscope.   The body as a whole was envisioned as an entirely interwoven entity.  Organic bodies were seen to exist as “living looms with regulating properties and a central weaving agent” in the words of one historian.  The fiber theory of living forms reigned throughout the 18th-century Enlightenment and into the first decades of the 19th century.  The cell theory of living forms replaced it beginning in the late 1830s thanks to the German scientists Theodor Schwann, Matthias Jakob Schleiden, and Rudolf Virchow, the father of cellular pathology. 

Rudolf Virchow

Virchow was inspired by the poet, writer, and scientist Johann Wolfgang von Goethe as recounted in his treatise “Goethe as a Natural Scientist.”  Goethe’s writings about and drawings of plant morphological patterns, what he called “the secret of plant generation and organization,” served as a backdrop to Virchow’s later microscopic explorations.   They also constituted a "proto-algorithmic matrix" that foretold computational pattern recognition in our time.  Goethe’s “Theory of Colors” (1840) in which he alluded to dyes used in tapestry was the culmination of what Virchow described as his “long term research into light and color” which was “not a lost effort,” certainly not for pathologists who soon employed dyes and stains to highlight cells and tissue structure for microscopic examination.  “Virchow, in a sense, gave Goethe's grand, philosophical ideas a concrete, empirical foundation that would revolutionize medicine and form the basis of modern pathology,” says Google’s AI tool Gemini.

In brief, several technological streams – computational, analytical, and biological – that are converging today actually began flowing concurrently nearly two centuries ago.  In England in 1843, Ada Lovelace published her “Note G,” a complex computer algorithm.  It was designed to calculate Bernoulli numbers using Babbage’s Analytical Engine, which actually was never finished.  In Germany, scientific visualization, pattern recognition, and organic generation were given theoretical foundations with Goethe’s optical, color, and plant studies.  Meanwhile, Schleiden, Schwann, and Virchow established that cells rather than fibers are the basic units of life.  Virchow issued his famous principle omnis cellula e cellula, “all cells come from cells,” after embryologist Robert Remak proposed in 1852 that cells divide yielding daughter cells.  Cells are the secret to the generative and regenerative processes and the organizational forms and patterns of living things that Goethe envisioned.

Decoding the cellular tapestry with AI to diagnose disease

 

photo of Paige's Virchow AI Foundation Model

Now let’s fast forward two centuries and bring Lovelace, Goethe, and Virchow along to the contemporary scene.  In 2024, Paige.ai in collaboration with Microsoft and several research institutions including Memorial Sloan-Kettering Cancer Center released a digital pathology foundation model, described as a pan-cancer detector, and named it Virchow.  “The analysis of histopathology images with artificial intelligence aims to enable clinical decision support systems and precision medicine,” authors of a paper about the model wrote in the journal Nature.  “The success of such applications depends on the ability to model the diverse patterns observed in pathology images. To this end, we present Virchow, the largest foundation model for computational pathology to date.”   The model was trained on 1.5 million H&E-stained whole slide images from 100,000 cancer patients.

Foundation models are deep, large-scale neural networks trained on vast datasets.  Virchow uses a “family” of self-supervised machine learning algorithms, meaning the model can train itself. Its power to recognize patterns and arrangements is extraordinary, animating Goethe's conceptual insights. The model can identify, delineate, and capture a broad spectrum of patterns including those associated with cell and nuclear morphology, tissue architecture, mitosis, necrosis, inflammatory response, neovascularization, and biomarker expression. The patterns in nature, ordered and disordered, are revealed in the human cellular tapestry.  Some of them, such as the arrangement of collagen fibers in breast cancer, can be predictive patterns, early signs of what lies ahead without intervention. 

Eric Topol is arguably the public face and professional voice of AI in American medicine.  Topol is a cardiologist, scientist, writer, and founder and director of the Scripps Research Translational Institute in La Jolla, California.   He has nearly 700,000 followers on Twitter / X, where he posts regularly about research advances in medicine and technology.   In 2014, with the publication of my article “The Shifting Currents of Bioscience Innovation” in a special “geotechnologies” issue of the journal Global Policy, I sent Topol a technological timeline figure from the paper at his request and he posted it approvingly on X with the comment “Progress in medicine is now happening so fast it can’t be accurately graphed!”  We’ve corresponded ever since -- about the book I coauthored with LMP head Leo Furcht (The Biologist’s Imagination:  Innovation in the Biosciences, Oxford, 2014) and subjects that I’ve written about for the LMP website including genomics and clinical medicine, spatial omics, and biomarkers in neurodegenerative disease.

“Pathologists have been much slower at adopting digitization of scans than radiologists—they are still not routinely converting glass slides to digital images and use whole-slide imaging (WSI) to enable viewing of an entire tissue sample on a slide,” Topol wrote in a review article published in Nature Medicine in 2019.  He noted the variability and inconsistency among pathologists’ interpretations of slides. “Deep learning of digitized pathology slides offers the potential to improve accuracy and speed of interpretation.”

Pathology:  The generative era of AI

Photo of NVIDIA Ada Lovelace GPU chip

Six years later, in 2025, Topol sees that things have changed.  In “The generative era of AI” published in the journal Cell by Topol with three colleagues, the authors write: “Multimodal AI is rapidly progressing in the field of pathology.  Large pathology training sets can incorporate standardized images of slides and specimens with text reports, genomic data, and EHR [electronic health record] data, in total providing a prime target for AI research.”  Topol and his colleagues see a future of transformer-enabled ‘‘vision-language’’ models like PathChat, models that incorporate text and image-analysis components including captioning into a multimodal AI assistant.  PathChat DX, which bills itself as an AI “copilot” with a built-in custom-trained multimodal large language model (MLLM), received U.S. Food and Drug Administration Breakthrough Device Designation early this year.

Pathology is at the heart of medical science, which means it is at the heart of AI’s potential in medical science and understanding the complex tapestry of diseases like cancer.  In 1852, the year Virchow delivered a series of lectures describing cellular pathology as “a theory of locally bound disease,” Ada Lovelace died at age 36, less than a decade after she wrote her Note G algorithm.  Virchow’s subsequent “all cells come from cells” pronouncement has a self-referential algorithmic aspect to it, as scholars have noted.  Consider that computer-generated “cellular automata” are used to model biological systems and features including self-assembly and pattern formation.  They bear a close relationship to the convolutional neural networks that enable AI.  Cells may not weave their support matrix, tissues, vessels, and organ systems like a Jacquard loom, but the resulting designs, patterns, and architectures from their evolutionary interactions flow in a current of multicellular life a billion years old.

Given a growing global population, rising urbanization, emerging pathogens spread by air travel, environmental toxins, and genetic disease, pathologies among the multicellular organism Homo sapiens are set to capture an expanding share of its destiny.  Can the powers of new technology alter that future?

Lovelace died from uterine cancer and was buried next to her father, Lord Byron.  Uterine or endometrial cancer is the fourth fastest-growing cancer among women worldwide. The median age of death from the disease in the U.S. today is 72, twice Lovelace’s age when she died.  More than 80 percent of patients live at least five years following diagnosis.  Earlier this year, an international research group reported that it had developed a model that can detect endometrial cancer with greater than 99 percent accuracy – a nearly 20 percent advance over other predictive models -- based on a deep learning algorithmic set analysis of thousands of histopathologic images. 

To borrow Lovelace’s words as she introduced Note G, “It is desirable to guard against the possibility of exaggerated ideas that might arise” concerning the powers of new technology.  It was a prudent caution then and especially now given the projected power of algorithms to shape human futures, complex procedural power scripted from cellular interactions in the human brain.  It is why pathologists insist that the tools they use are properly validated before signing out a final patient report.  After all, AI models are only as good as their training sets allow them to be.  Yet there is no escaping that the marvelous and ever-present electronic heirs of the imaginary, steam-powered Analytical Engine that Ada Lovelace programmed have put us on a generative path in medical science, with pathology at its heart.   

We are just at the beginning.

Here are the stories in our DP & AI series: