Blog

My Random Thoughts 

Deep Learning and AI: Revisit Francis Crick                    

During 2019 Chinese Lunar Year Festival, I was fascinated and intrigued by Character of Francis Crick and his research method. To prevent indulgence, I decide to write a summary my thoughts and move on coding on  Tensorflow, Pytorch and paddlepaddle.   

on Crick’s deep learning and research method and style

 

Thoughts from Hiwave Lab

2/8/2019

***

The first sentence of James Watson’s Popular Book entitled Double helix states: I have never seen Francis Crick in a modest mood.

James Watson has been a house hold name in American. Every one knows that it is James Watson who discovered DNA ‘s Double Helix structure.

But Who is Francis Crick?

The protagonist of brilliant book << Brave Genius>>, Jacques Monod has a brief statement: No Man discovered or created molecular biology. But one man dominates intellectually the whole field, because he knows the most and understands the most. Francis Crick.

Crick used to sail with Monod. Monod echoes Watson that he had only seen Francis Crick in a modest mood was watching Crick nervously bring a boat into harbor. Crick admired Monod’s sailing skill and Monod’s polymath talent, “The Scientist, the philosopher, the man of action and the musician”, as Crick remembered Monod. But only one earned this brave genius’s utmost respect: for his domination intellectually the whole molecular biology field, that was Francis Crick.

Indeed, from the day James Watson met Crick at Cavendish Cambridge in September 1951 to 1976 of Crick’s retirement from Medical Research Center at Cambridge, Francis Crick dominated intellectually the whole molecular biology field.

What happen to Francis Crick after his retirement in 1976? Did he finish? Did he vanish from his brilliant scientific research Career?

***

   Once upon a time, most likely a winter, there was a field called Artificial Intelligent.

Timeline:

1950 --- Alan Turning published his seminal paper: Computing Machinery and Intelligence”. A question proposed by Alan Turing: Can a Machine imitate human intelligence?

1956 --- Dartmouth professor John McCarthy organized a Conference to “explorer ways to make a machine that could reason like a human, was capable of abstract thought, problem-solving and self-improvement”. John McCarthy first coined the term “ Artificial Intelligence” and it was at this conference where AI gains its Vision, mission and Hype. John McCarthy also created the LISP Language. He won the Turing award in 1971.

1956 --- Founding fathers of AI: John McCarthy, Marvin Minsky, Allen Newell and Herbert Simon.

1962 --- Frank Rosenblatt Published 626 pages Book entitled Principles of Neurodynamics: Perceptrons and the Theory of Brain Mechanisms.  (If you read today’s Modern Machine learning/Deep Learning and Nerul Network books, you would be amazed by how harbingering this book was: cost $274.95 in Amazon.com)

1962 --- David Hubel and Torsten Wiesel published paper entitled Receptive Fieldsm Binocular interaction and Functional Architecture in the Cat’s Visual Cortex. Deep Learning networks have an architecture similar to hierarchy of areas in the visual cortex.

1969 --- Marvin Minsky and Seymour Paper published Perceptrons. Which marked the beginning of a neural network winter by stated the computational limitations of a single artificial neuron.

1979  --- During the first AI winter, Many left the field, Geoffrey Hinton stayed.  Geoffrey Hinton and James Anderson Organized the Parallel Models of Associative Memory workshop in La Jolla, California.  Francis Crick was among audiences.

1985 --- Geoffrey Hinton and Terry Sejnowski publish “A Learning Algorithm for Boltzmann Machines”. Counterexample to Marvin Minsky’s belief that no learning algorithm for multilayer networks was possible. Hinton recall later that this work is his most rewarding/joyful work

1987 --- The first Neural Information System (NIPS) Conference and Workshop.

2006 --- Hinton et. Al publish paper entitled A fast learning algorithm, for Deep belief nets. Hence coined the term “Deep Learning”.  The Machine Learning in General, Deep Learning in Particular incarnates.

2015 --- LeCun (Facebook and NYU), Bengio (U of Montreal) and Hinton(Google and U of Toronto)  co-authored Paper “Deep Learning” in Nature. Symbolically established as Fathers of Deep Learning or so called three-omega of Deep Learning.

2017 --- AlphaGo a deep learning network program beats the world champion at Go. AI, ML, and DL Tsunami begins.

    Fancies Crick brought Terry Sejnowski to Salt Institute. Terry Sejnowski has worked closed with Hinton.      ( So path like CrickàSejnowskiàHinton model as DNA --> RDNAàProtein )

 

***

No, Francis did not finish.

But Francis Crick was not a Computer Scientist like Hinton, Bengio and LeCun, nor Computation Neuroscientist like Terry Sejnowski.

Francis Crick was probably the greatest theoretical Biologist of twentieth century.

After dominated intellectually the whole molecular biology for 20 years, Crick needed to change a new field to explorer and ponder…

Crick suddenly recall his childhood dreams.  When he was in childhood, Crick has two dreams: to crack the 1) The Secrete of Life; 2) The secret of Brain.

After discovery the secret of the life by cracking the DNA double Helix and genetic sequence genes, at age 60th, he decided to study the secret of Brain, how brain conscious work and how brain thinks.

Sounds like Artificial Intelligent? No, it is studying of Natural Stupidity.

Francis has a peculiar learning and research method, the method its self could be called deep learning.  In physics, theoretician called theoretical physicist, experimentalist called experimental physicist. But in biology, there is no such division. All biologists are expected to conduct experiment skillfully, But Francis Crick (as well as James Walton).

Crick’s learning and research method are quite thinking, massive reading, and Socrates Dialog. IBM research has a term called World is our laboratory. For Francis Crick, all top Biological Laboratories are his Laboratory. He would study the massive data produced by the top experimentalist and from which he would induct a theory and further abstract thinking, more, he would build a model. Sometime, he would propose a theory and many top laboratories would go ahead to conduct experiments to prove if Crick’s theory is true or false. Sometime works sometime not. When it works, the theory is brilliant and the model is beautiful such as DNA double Helix.

There are four scientific partners and intellectual sound boards in Crick’s productive career. They are George Kreisel, James Walton, Sydney Brenner and Christof Koch. They functioned as Chief Conversation Officer of Crick. The main job is to have had Socrates dialog with Francis Crick, from morning to night, day to day, every day, intensively. Even Koch was wondering one day after 4 hours intensive conversation, how Crick’s wife survived this intensity 10 hours a day 7 days a week and sustained for over 50 years of their companionship.

The first intellectual sound board of Crick is George Kriesel. George Kriesel was a formidable mathematical logician who made profound contributions to proof theory. Here is what Francis Crick had to say: “When I met Kriesel I was very sloppy thinker. His powerful, rigorous mind gently but steadily made my thinking more incisive and occasionally more precise. Quite a number of my mental mannerisms spring from him”.

Then came with James Walton with whom they co-discovered DNA Double Helix Structure. “If Walton had been killed by a tennis ball, (Walton has been a avid tennis player), I am reasonably sure I would not have solved DNA structure alone.”

Sydney Brenner was Crick’s closest associate for almost 20 years, they shares same office almost 20 years. During much of that that time, they had long scientific discussions on almost every working day. His clarity, incisiveness, and fertile enthusiasm made him an ideal colleague.

The last one was Christof Koch. From 1986 to his death, 2004, 18 years, they had very intensive conversations despite their age difference. Before his death, Crick gave a life side picture of his to Koch, caption: “I am watching you”. Koch left Caltech to join Paul Allen Brain Institute, first as Chief Scientific Officer, last as its President. (Without Crick, Koch himself without a intellectual sound board, hence the administrator).

In Barcelona, there had/has been two persons, only two people. One created history, one has been creating history day by day. The former was architect Gaudi, once one gets out of the airport, one can see Gaudi’s Architecture everywhere. The other is Messi. Oh Messi, that GOAT!

In San Diego, California, there has been two people wrote the history, One is Jacobs who created QUALCOMM, whose chip had early  powered every Iphone. The other was Francis Crick, Oh, Crick that GOAT. For him, San Diego affectionately called Neuron Valley.  To bestow them, Salk institute establishes a Crick-Jocobs Center for Computational Neurobiology.

Francis Crick also set up an example for next generation scientists. He was never in a modest mood, but he was hardly at arrogant. He was generous at his time for young scientists. He was approachable and always thirst for new ideas and new blood. When he had disagreement with Walton, he was never one to bear a grudge. Since he wanted to set an example for next generation’s pair collaboration. When he was seriously ill, he still fully devoted to science he had loved. At Salk institute, at his home and his Hospital bed. He had good days and bad days, but his eye still let him read and his brain still let him think, so there was no reason not to work. Odile Crick (His French Wife) looked to his comforts, day and night, as she had done for 55 years, There was time for one more idea: ab obscure brain structure called the CLUSTRUM might be the source of a striking feature of consciousness, integrated unit. Like Fermi, before his final hour, he still modified his manuscript and conducted Socrates dialog with Christof Koch one more last time.

The British did not forget their favorite son, Francis Crick, Greatest Molecular Biologist of the 20th Century, and perhaps greatest scientist since Newton, Darwin and Dirac the British has to offer.  In 1 Midland, London, the British build Francis Crick Institute, with its beautiful modern building, top scientists, it is one of top 3 life science center in the world.  The people of the institute affectionately called Francis Crick Institute, The Crick.

Hence, Francis Crick, the Crick.

I closed the book, sitting on the floor in a Yoga position, my eye closed, meditation music arises … {in 20th Century science, Italy Pops Fermi, American produces Feynman, Hungary donates John von Neumann, China contributes Yang, Britain gifts Crick}

***

Why Crick? I am asking myself?

From his <<What mad Pursuit: A personal View of Scientific Discovery>>, it seems embodied the answer:

Part of the trouble with theoretical neuroscience is that it lies somewhat between THREE other fields. At one extreme we have those researchers working directly on the brain. This Science. It is attempting to discover what devices nature actually uses. At the other extreme lies Artificial Intelligence. This is Engineering. Its object is to produce a device that works in the desired way. The third field is mathematics. Mathematics cares neither for science nor for engineering (except as a source or problems) but only about the relationship between abstract entities.  If an idea they conceive doesn’t help to explain the brain, the theorists may hope that perhaps it may be useful in AI. There is thus no compelling drive for the to press on and on until the way the brain actually works laid bare. It is more fun to produce for such “interesting “Computer programs and much easier to get grants for such work. There is even the possibility that they might make some money if their ideas could be used un Computers.  (That was in 1988).

Hence The Google’s Tenserflow and Keras, Facebook’s Pytorch and Caffe 2 ,Microsoft’s CNTK, Baidu’s PaddlePaddle.  The flood gate opened in 2015, so began the Deep Learning Revolution…

 

February 8, 2019 Friday

Old Farm Lake ( 老农湖畔)

Matthew Cobb, Life’s Greatest Secret: The Race to Crack the Genetic Code, 2015, Basic Books, Inc. Publishers.

Francis Crick, What Mad Pursuit: A Personal View of Scientific Discovery, 1988, Basic Book Inc. Publishers.

Matt Ridley, Francis Crick: Discoverer of The Genetic Code, 2006, Eminent Lives Series, HarperCollins Publishers.

Sean Carroll, Brave Genius, 2013, Crown Publishers.

James Walton, Double Helix, Edited by Alexander Gann & Jan Witkowsk, New Annotations, Illustrations and Appendixes, 2012, Simon & Schuster HardCover Edition.

James Watson, DAN the Story of the Genetic Revolution, Newly Revised and Updated, 2017, Alfred Knopd, Publisher.  “For Francis Crick”.

Terry Sejnowski, Deep Learning Revolution, 2018.