Blog

My Random Thoughts 

Deep Learning and AI: Revisit Francis Crick                    

During 2019 Chinese Lunar Year Festival, I was fascinated and intrigued by Character of Francis Crick and his research method. To prevent indulgence, I decide to write a summary my thoughts and move on coding on  Tensorflow, Pytorch and paddlepaddle.   

on Crick’s deep learning and research method and style

 

Thoughts from Hiwave Lab

2/8/2019

***

The first sentence of James Watson’s Popular Book entitled Double helix states: I have never seen Francis Crick in a modest mood.

James Watson has been a house hold name in American. Every one knows that it is James Watson who discovered DNA ‘s Double Helix structure.

But Who is Francis Crick?

The protagonist of brilliant book << Brave Genius>>, Jacques Monod has a brief statement: No Man discovered or created molecular biology. But one man dominates intellectually the whole field, because he knows the most and understands the most. Francis Crick.

Crick used to sail with Monod. Monod echoes Watson that he had only seen Francis Crick in a modest mood was watching Crick nervously bring a boat into harbor. Crick admired Monod’s sailing skill and Monod’s polymath talent, “The Scientist, the philosopher, the man of action and the musician”, as Crick remembered Monod. But only one earned this brave genius’s utmost respect: for his domination intellectually the whole molecular biology field, that was Francis Crick.

Indeed, from the day James Watson met Crick at Cavendish Cambridge in September 1951 to 1976 of Crick’s retirement from Medical Research Center at Cambridge, Francis Crick dominated intellectually the whole molecular biology field.

What happen to Francis Crick after his retirement in 1976? Did he finish? Did he vanish from his brilliant scientific research Career?

***

   Once upon a time, most likely a winter, there was a field called Artificial Intelligent.

Timeline:

1950 --- Alan Turning published his seminal paper: Computing Machinery and Intelligence”. A question proposed by Alan Turing: Can a Machine imitate human intelligence?

1956 --- Dartmouth professor John McCarthy organized a Conference to “explorer ways to make a machine that could reason like a human, was capable of abstract thought, problem-solving and self-improvement”. John McCarthy first coined the term “ Artificial Intelligence” and it was at this conference where AI gains its Vision, mission and Hype. John McCarthy also created the LISP Language. He won the Turing award in 1971.

1956 --- Founding fathers of AI: John McCarthy, Marvin Minsky, Allen Newell and Herbert Simon.

1962 --- Frank Rosenblatt Published 626 pages Book entitled Principles of Neurodynamics: Perceptrons and the Theory of Brain Mechanisms.  (If you read today’s Modern Machine learning/Deep Learning and Nerul Network books, you would be amazed by how harbingering this book was: cost $274.95 in Amazon.com)

1962 --- David Hubel and Torsten Wiesel published paper entitled Receptive Fieldsm Binocular interaction and Functional Architecture in the Cat’s Visual Cortex. Deep Learning networks have an architecture similar to hierarchy of areas in the visual cortex.

1969 --- Marvin Minsky and Seymour Paper published Perceptrons. Which marked the beginning of a neural network winter by stated the computational limitations of a single artificial neuron.

1979  --- During the first AI winter, Many left the field, Geoffrey Hinton stayed.  Geoffrey Hinton and James Anderson Organized the Parallel Models of Associative Memory workshop in La Jolla, California.  Francis Crick was among audiences.

1985 --- Geoffrey Hinton and Terry Sejnowski publish “A Learning Algorithm for Boltzmann Machines”. Counterexample to Marvin Minsky’s belief that no learning algorithm for multilayer networks was possible. Hinton recall later that this work is his most rewarding/joyful work

1987 --- The first Neural Information System (NIPS) Conference and Workshop.

2006 --- Hinton et. Al publish paper entitled A fast learning algorithm, for Deep belief nets. Hence coined the term “Deep Learning”.  The Machine Learning in General, Deep Learning in Particular incarnates.

2015 --- LeCun (Facebook and NYU), Bengio (U of Montreal) and Hinton(Google and U of Toronto)  co-authored Paper “Deep Learning” in Nature. Symbolically established as Fathers of Deep Learning or so called three-omega of Deep Learning.

2017 --- AlphaGo a deep learning network program beats the world champion at Go. AI, ML, and DL Tsunami begins.

    Fancies Crick brought Terry Sejnowski to Salt Institute. Terry Sejnowski has worked closed with Hinton.      ( So path like CrickàSejnowskiàHinton model as DNA --> RDNAàProtein )

 

***

No, Francis did not finish.

But Francis Crick was not a Computer Scientist like Hinton, Bengio and LeCun, nor Computation Neuroscientist like Terry Sejnowski.

Francis Crick was probably the greatest theoretical Biologist of twentieth century.

After dominated intellectually the whole molecular biology for 20 years, Crick needed to change a new field to explorer and ponder…

Crick suddenly recall his childhood dreams.  When he was in childhood, Crick has two dreams: to crack the 1) The Secrete of Life; 2) The secret of Brain.

After discovery the secret of the life by cracking the DNA double Helix and genetic sequence genes, at age 60th, he decided to study the secret of Brain, how brain conscious work and how brain thinks.

Sounds like Artificial Intelligent? No, it is studying of Natural Stupidity.

Francis has a peculiar learning and research method, the method its self could be called deep learning.  In physics, theoretician called theoretical physicist, experimentalist called experimental physicist. But in biology, there is no such division. All biologists are expected to conduct experiment skillfully, But Francis Crick (as well as James Walton).

Crick’s learning and research method are quite thinking, massive reading, and Socrates Dialog. IBM research has a term called World is our laboratory. For Francis Crick, all top Biological Laboratories are his Laboratory. He would study the massive data produced by the top experimentalist and from which he would induct a theory and further abstract thinking, more, he would build a model. Sometime, he would propose a theory and many top laboratories would go ahead to conduct experiments to prove if Crick’s theory is true or false. Sometime works sometime not. When it works, the theory is brilliant and the model is beautiful such as DNA double Helix.

There are four scientific partners and intellectual sound boards in Crick’s productive career. They are George Kreisel, James Walton, Sydney Brenner and Christof Koch. They functioned as Chief Conversation Officer of Crick. The main job is to have had Socrates dialog with Francis Crick, from morning to night, day to day, every day, intensively. Even Koch was wondering one day after 4 hours intensive conversation, how Crick’s wife survived this intensity 10 hours a day 7 days a week and sustained for over 50 years of their companionship.

The first intellectual sound board of Crick is George Kriesel. George Kriesel was a formidable mathematical logician who made profound contributions to proof theory. Here is what Francis Crick had to say: “When I met Kriesel I was very sloppy thinker. His powerful, rigorous mind gently but steadily made my thinking more incisive and occasionally more precise. Quite a number of my mental mannerisms spring from him”.

Then came with James Walton with whom they co-discovered DNA Double Helix Structure. “If Walton had been killed by a tennis ball, (Walton has been a avid tennis player), I am reasonably sure I would not have solved DNA structure alone.”

Sydney Brenner was Crick’s closest associate for almost 20 years, they shares same office almost 20 years. During much of that that time, they had long scientific discussions on almost every working day. His clarity, incisiveness, and fertile enthusiasm made him an ideal colleague.

The last one was Christof Koch. From 1986 to his death, 2004, 18 years, they had very intensive conversations despite their age difference. Before his death, Crick gave a life side picture of his to Koch, caption: “I am watching you”. Koch left Caltech to join Paul Allen Brain Institute, first as Chief Scientific Officer, last as its President. (Without Crick, Koch himself without a intellectual sound board, hence the administrator).

In Barcelona, there had/has been two persons, only two people. One created history, one has been creating history day by day. The former was architect Gaudi, once one gets out of the airport, one can see Gaudi’s Architecture everywhere. The other is Messi. Oh Messi, that GOAT!

In San Diego, California, there has been two people wrote the history, One is Jacobs who created QUALCOMM, whose chip had early  powered every Iphone. The other was Francis Crick, Oh, Crick that GOAT. For him, San Diego affectionately called Neuron Valley.  To bestow them, Salk institute establishes a Crick-Jocobs Center for Computational Neurobiology.

Francis Crick also set up an example for next generation scientists. He was never in a modest mood, but he was hardly at arrogant. He was generous at his time for young scientists. He was approachable and always thirst for new ideas and new blood. When he had disagreement with Walton, he was never one to bear a grudge. Since he wanted to set an example for next generation’s pair collaboration. When he was seriously ill, he still fully devoted to science he had loved. At Salk institute, at his home and his Hospital bed. He had good days and bad days, but his eye still let him read and his brain still let him think, so there was no reason not to work. Odile Crick (His French Wife) looked to his comforts, day and night, as she had done for 55 years, There was time for one more idea: ab obscure brain structure called the CLUSTRUM might be the source of a striking feature of consciousness, integrated unit. Like Fermi, before his final hour, he still modified his manuscript and conducted Socrates dialog with Christof Koch one more last time.

The British did not forget their favorite son, Francis Crick, Greatest Molecular Biologist of the 20th Century, and perhaps greatest scientist since Newton, Darwin and Dirac the British has to offer.  In 1 Midland, London, the British build Francis Crick Institute, with its beautiful modern building, top scientists, it is one of top 3 life science center in the world.  The people of the institute affectionately called Francis Crick Institute, The Crick.

Hence, Francis Crick, the Crick.

I closed the book, sitting on the floor in a Yoga position, my eye closed, meditation music arises … {in 20th Century science, Italy Pops Fermi, American produces Feynman, Hungary donates John von Neumann, China contributes Yang, Britain gifts Crick}

***

Why Crick? I am asking myself?

From his <<What mad Pursuit: A personal View of Scientific Discovery>>, it seems embodied the answer:

Part of the trouble with theoretical neuroscience is that it lies somewhat between THREE other fields. At one extreme we have those researchers working directly on the brain. This Science. It is attempting to discover what devices nature actually uses. At the other extreme lies Artificial Intelligence. This is Engineering. Its object is to produce a device that works in the desired way. The third field is mathematics. Mathematics cares neither for science nor for engineering (except as a source or problems) but only about the relationship between abstract entities.  If an idea they conceive doesn’t help to explain the brain, the theorists may hope that perhaps it may be useful in AI. There is thus no compelling drive for the to press on and on until the way the brain actually works laid bare. It is more fun to produce for such “interesting “Computer programs and much easier to get grants for such work. There is even the possibility that they might make some money if their ideas could be used un Computers.  (That was in 1988).

Hence The Google’s Tenserflow and Keras, Facebook’s Pytorch and Caffe 2 ,Microsoft’s CNTK, Baidu’s PaddlePaddle.  The flood gate opened in 2015, so began the Deep Learning Revolution…

 

February 8, 2019 Friday

Old Farm Lake ( 老农湖畔)

Matthew Cobb, Life’s Greatest Secret: The Race to Crack the Genetic Code, 2015, Basic Books, Inc. Publishers.

Francis Crick, What Mad Pursuit: A Personal View of Scientific Discovery, 1988, Basic Book Inc. Publishers.

Matt Ridley, Francis Crick: Discoverer of The Genetic Code, 2006, Eminent Lives Series, HarperCollins Publishers.

Sean Carroll, Brave Genius, 2013, Crown Publishers.

James Walton, Double Helix, Edited by Alexander Gann & Jan Witkowsk, New Annotations, Illustrations and Appendixes, 2012, Simon & Schuster HardCover Edition.

James Watson, DAN the Story of the Genetic Revolution, Newly Revised and Updated, 2017, Alfred Knopd, Publisher.  “For Francis Crick”.

Terry Sejnowski, Deep Learning Revolution, 2018.

 

 

 

 

摄影和诗

摄影和诗

作者:廸尔海波

 

(一) 疑是银河落九天

摄影就是用镜头讲故事。摄影的最高境界是诗。

摄影家有善长用广角镜头和善长用聚焦镜头的。美国著名摄影家,少壮派John Paul CaponigroSeth Resnick就是例子。在每年解伴在冰岛拍摄时,一人拍24mm 以下的广角画面,一人拍200mm 以上的聚焦画面。拍出迥异不同风格的作品。

诗人也有善长用广角镜头和善长用聚焦镜头的。玩广角镜头最好的首推李白了。让我们一起看看他用14mm广角镜头拍出的画面:

日照香炉生紫烟,遥看瀑布挂前川。 
飞流直下三千尺,疑是银河落九天

疑是银河落九天?  这个14mm广角镜头用得何等之好,令多少代人拍案叫绝!

 

(二)日光岩下的三角梅

鼓浪屿上有一个石头,经过岁月的冲洗,变得巨大无比,俗称"岩仔山",别名"晃岩".  相传1641年,郑成功来到晃岩,看到这里的景色胜过日本的日光山,便把""字拆开,称之为"日光岩"Sunlight Rock)。

日光岩是鼓浪屿的最高点。登上日光岩, 便可360度的view 全岛和全厦门乃金门岛。

这可是玩广角镜头绝好地点。

但本岛诗人舒婷却用了个200mm 的聚焦特写镜头,令我拍案叫绝!

让我们看看舒婷用200mm 的聚焦特写镜头,拍出的画面:

    日光岩下的三角梅

作者:舒婷


  是喧闹的飞瀑 
  披挂寂寞的石壁 
  最有限的营养 
  却献出了最丰富的自己 
  是华贵的亭伞 
  为野荒遮蔽风雨 
  越是生冷的地方 
  越显得放浪、美丽 
  不拘墙头、路旁 
  无论草坡、石隙 
  只要阳光常年有 
  春夏秋冬 
  都是你的花期 
  呵,抬头是你 
  低头是你 
  闭上眼睛还是你 
  即使身在异乡他水 
  只要想起 
  日光岩下的三角梅 
  眼光便柔和如梦 
  心,不知是悲是喜

 1979.8


  让还记得篇一的李白的14mm广角镜头下的飞瀑吗?对,疑是银河落九天的飞瀑!舒婷引喻了李白开头,但用了200mm 的聚焦特写镜头切入,拍出了的披挂寂寞的石壁上,日光岩下的三角梅的画面。

         春夏秋冬 
  都是你的花期 
  呵,抬头是你 
  低头是你 
  闭上眼睛还是你 

让我们一起看看住在诗人舒婷隔壁的李敖的一首诗:

        不看你的眼,

        不看你的眉,

        看了心里都是你,

        忘了我是谁。

真是有异曲同工之妙。

 

(三)留在罗马的詩

谈起李敖,写得最好的记念文章,当首推刘长乐追忆李敖先生:怎能忘了你是谁?

谈起罗马,立马就有条条道路同罗马。

有谁见过玩得好的广角镜头和聚焦镜头下的罗马的诗?

平心而论,廸尔海波的留在罗马的詩似乎勇敢地填着空白。

让我们看看廸尔海波用24mm 的广角镜头,拍出的画面:

 

                                  留在罗马的詩

                                作者:廸尔海波

 我迈著沉重的步子

 漫遊在罗马的古城

 一步一年

 走了二千零八十五步

 

 那巨大椭园的竞技场

 仍闪烁着昔日的辉煌

 那城心的残垣断壁

 没有八国联军的摧残

 怎恰似东方的圆明园?

 

 十步一问

 问向西班牙广场

 凯撒奥古斯都大帝的威武不再

 铁蹄中寻不出复兴的大旗

 只有那日夜不停的喷泉水

 

 我吸吮着古罗马清鲜的空气

 呼吐出一道历史长河

 千年一叹

 叹了二声半?

 2015.6.8

 罗马 
 

怎恰似东方的圆明园?

铁蹄中寻不出复兴的大旗

呼吐出一道历史长河

好强的气场,谁有这气派? 

 

(四)

有!独领风骚者也!

某年某月某冬日,毛泽东来到西湖。下榻在距笔者今天下榻地直线距里不远的西湖宾馆。

两部镜头:广角镜头和聚焦镜头;

拍摄对象:梅

毛泽东拿起广角镜头, 又拿起聚焦镜头,瞄了下拍摄对象。

梅, 多少代千古文人摄影家都拍摄过。

我如何拍出新鲜的画面?

毛泽东叫来田家英:找来陆游的作品,看看他是如何玩的?

田家英找来陆游作品。

毛泽东看了看画面。

毛泽东敏感地意识到:陆游虽非等闲之辈,但便不是摄影大师:黄昏光线本来就不好, 还加filter ? 只有“只有香如故”味道不错 (却见人去楼空,唯有书海留香)。可是我们是搞摄影的。摄影是闻不出香味的啊?

我们来看陆游是如何拍摄梅的。咏梅

 

卜算子· 咏梅

                                作者:陆游

驿外断桥边,寂寞开无主。已是黄昏独自愁,更著风和雨

无意苦争春,一任群芳妒。零落成泥碾作尘,只有香如故

 

看了陆游的画面,独领风骚者果断地丢掉filter, 择了一个有阳光的日子。反其意而作,开始了自己的拍摄。

让我们在入睡前,来再一次看看毛泽东是如何拍摄梅的:

卜算子· 咏梅

                                作者:毛泽东

风雨送春归,飞雪迎春到。已是悬崖百丈冰,犹有花枝俏。

俏也不争春,只把春来报。待到山花烂漫时,她在丛中笑。

 

朋友,当你看到悬崖百丈冰的梅时,你是否联想到舒婷披挂寂寞的石壁上,日光岩下的三角梅?

 

 

 

廸尔海波

2018.5.25 2:18am 于西子湖畔 第一稿

留在罗马的詩

留在罗马的詩

 (2016-01-01 20:10:14)

 

【留在罗马的詩】

        滴尔海波

 

  我迈著沉重的步子

        漫遊在罗马的古城

          一步一年

          走了二千零八十五步

   那巨大椭园的竞技场

       仍闪烁着昔日的辉煌

          那城心的残垣断壁

         没有八国联军的摧残

     怎恰似东方的圆明园?

 

          十步一问

          问向西班牙广场

    恺撒奥古斯都大帝的威武不再

       铁蹄中寻不出复兴的大旗

        只有那日夜不停的喷泉水…

 

     我吸吮着古罗马清鲜的空气

          呼吐出一道历史长河

          千年一叹

          叹了二声半?

 

          二零一五年六月八日

          罗马

Best Black Friday and Discovery of Ernst Haas

A few year ago, I checked a book out from local library, entitled <<How to look at Photographs>> by David Finn. It is an excellent book. I did not purchase it, because the rating on Amazon was not that :good:. But I read it and re-read it I can't put it down. so that I kept renew and returned it and checked it out.  I liked it do much I decided to own the book in my photographic library. Finally, I purchased the book from Amazondotcom, cost $0.25 plus shipping $3.99. That was a best Black Friday deal!

Why the book so intrigued me? Here are the few chapters: Discovery, Inner Spirit, outer visage, revelations, the natural world, moment of being.  on p. 34, it discussed Henri Cartier-Bresson's Place de l'Europe Paris 1932. Who believed that what was seen in that decisive moment was the essence of photography. His eye was always on the lookout for things to come together: Sometimes you have the feeling that here are all the makings of a picture--except for just one thing that seems to be missing. But what one thing? Perhaps someone suddenly walks into your range of view. You follow his progress through the viewfinder. You wait and wait, and then finally you press the button -- and you depart with the feeling (thought you don't know why) that you've really got something. Wow! Yes I have had that feeling. 

I knew Henri Carti-Bresson's work. SO it is not a surprise. 

The surprise this book brought me was that it let me "Discovered" Ernst Haas. That feeling of discovery was so strong that It is so hard to express in writing:  I will try  next time.

The book made me believe and desire  to  be like Haas!

 

Cinematic EYE and Uncanny HEARING :Creative Giants

This week, Two Creative Giants in Cinematic Scene fell.  Both Giants trees marks its rings age at 83. 

        One wad Ken Takakura, 高倉 健, Japanse Film Actor, A popular Icon both in China and Japan. Here is what the New York Times has to say.

       The other was Film Director, or Legendary Directors (Because he was not only film director, but, music, Broadway and TV director), Mike Nichols. The husband of former ABC News Anchor Diane Sawyer.  Here is the sweet love story about two.

      What made so special which attracted absolutely gorgeous, smartest interviewer, kindest soul in Diane Sawyer?

      Here is what film director Steve Spielberg has to say: "Mike was a friend, a muse 缪斯, a mentor, one of America's all time greatest film and stage directors, and one of the most generous people I have ever known. For me, 'The Graduate' was life altering—both as an experience at the movies as well as a master class about how to stage a scene," 

      Steve  is insightful:  "Mike had a brilliant cinematic eye and uncanny hearing for keeping scenes ironic and real!"

      Director Ron Howard gives the answer: "Any conversation was rich with Wit and Wisdom"

      No wonder Mike Nichols' "Beautiful, Rich Wife", As he called, Diana Sawyer wad not bored after 26 years of marriage! 

     "Greatest of great!": A life well lived. 

Two greatest of great!  RIP!