How Claude Shannon’s Concept of Entropy Quantifies Information | Quanta Magazine

This concept forms the basis of Information Theory. In digital communications, it places an absolute limit on the maximum possible rate at which data can be transmitted on a channel reliably. This limit is theoretical, and the challenge is to implement coding schemes that can get us closer and closer to such limits in real systems.

How Claude Shannon’s Concept of Entropy Quantifies Information | Quanta Magazine

How Claude Shannon’s Concept of Entropy Quantifies Information | Quanta Magazine

A Reflection on “Deep Learning Poised to ‘Blow Up’ Famed Fluid Equations | Quanta Magazine”

For some reason or the other, this article took me mentally to a place of familiarity and comfort, of having “been there, done that”. This happened even though I have no experience with the subject of fluid dynamics – beyond perhaps the faint memories of a course or two taken during my undergraduate studies. The phrase “Navier-Stokes equations” sounds vaguely familiar. And I do have friends who worked on fluid dynamics for their doctoral theses.

I think that the way my brain is wired, I have tended to approach problem-solving in the past in general with a similar overall philosophy of attack as that which comes through in the article. One should be adaptable in the solution approach. One should not be afraid of patching together unconventional and unorthodox approaches for problem solving. Blindly following formulae can sometimes only get you so far. I have sometimes thought my thought process when visualizing problems and solutions did not necessarily belong in the mainstream, but perhaps that is not completely true these days.

However, having said all this, one should not make the mistake of underestimating the utility of characterizing processes using standard approaches, including the use of equations (or formulae) as applicable, to simplify problem solving. Many of our technological advances would not have been possible without recognizing a structure behind certain phenomena or observations, often stated in the form of equations. For example, without the use of Newton’s Laws of Motion, and the equations that follow, we would not be able to shoot and direct objects to the distant recesses of Space – and direct these objects precisely to where we want them to be at particular time instants – to be able to learn more about our place in the scheme of Everything. We used computers initially to speed up our use of these equations. Even during my time we could be innovative about the the use of computers for the application of these equations. I could actually come up with evaluation and analysis tools using techniques and concepts like iteration, convergence, equilibrium, intersection points, even educated guesses for solution spaces, etc.., based on standard equations that could be implemented on computers – which could lead to solution spaces that would not easily be accessible by hand. But we are now getting past even that stage. We are making use of the power of the computers in other ways for discovering and intuiting the structures themselves, and then using these structures to further problem solving.

As time passes, we also learn that many of the equations that we depend upon in our daily lives may be approximations of what is actually physically going on in the Universe. We still have not connected all the four “fundamental” forces of nature that we recognize today in a logical way. The relevant equations that characterize the individual forces do not connect with each other. There is something missing. Something else is going on.

As another aside, how many of us were aware that the GPS system that we have come to depend upon to find our locations on Earth would be inaccurate without accounting for the physical impact of the Theory of Relativity. Yes, the speed of light does have an impact in a practical sense in our lives even if we are not aware of it, and Newton’s equations by themselves are not adequate for some applications even though they would suffice for others. Pause for a minute to ponder the fact that Euclidean geometry, a tool that seems to work so well for us in characterizing our physical world, is not the only kind of geometry that exists, and that there are other kinds of geometries that obey different rules that can be useful for studying and understanding different systems and applications, and perspectives.

Coming back to simpler things like fluid flows, the subject of the attached article, there have been equations that have formed the basis of our understanding of fluid dynamics for years. These equations help us predict behaviors and design systems. But then we also learn from the article that there may be singularities, and the possible existence of a place or space where the equations do not quite add up (pun intended). The equations can actually break down. What do you do when the equations blow up, perhaps as one or more of the parameters tends towards zero and the solution tends to infinity? Do we need to push ourselves to try provide structure, perhaps in the form of equations, to our understanding everywhere, including at perceived or real singularities? In my mind, it only makes sense to the extent that these formulations help us to predict and foresee how real physical (and other) phenomena will play out in these “extreme” circumstances, thereby expanding our knowledge of the world, or creating the possibility of some practical application that may serve our benefit. I admit that I myself have a curiosity about what lies beyond the singularity of a Black Hole.

Enter now the new techniques and concepts like artificial intelligence, deep learning, neural networks, etc.., aided and abetted by the availability of massive data collection and storage capability, massive processing power, and even the emergence of innovative technologies like quantum computing. Essentially, you can gather up all your observations and process the data using your computer until you are at point where you can try to see a pattern (or structure). It may be a pattern that is easy to pick up on, or it may be something that is weak and difficult to follow. So you continue the computing process for as long as is necessary. With the current state of our technology, this process may be easier than trying to mentally discern a a structure or process that lies hidden, something that is not obvious to the common eye. After all, how many of us have the intuition of an Issac Newton to get to figure out concepts like Gravity?

The general approach described in the article below seems to me to be a combination of multiple techniques in what could be considered by some to be a somewhat unorthodox manner. The approach follows a methodology and a way of thinking that I actually feel quite comfortable with. The approach seems quite intuitive to me, and its usage would come naturally to me. Although I have no experience with the deeper science of artificial intelligence and deep learning, I would be quite happy making use of these tools as applicable and available to further whatever research or analysis that I might happen to find myself involved in. I would instinctively tend to look at the whole problem holistically without necessarily constraining my thinking to some misguided notion to purity of procedure.

The question that still bugs me regarding the article below is that of the reason why significant brainpower and computer resources are being expended on this particular topic. Why are people so intent on “Blowing Up” the equations? Is it just intellectual curiosity (dare I call it idle curiosity), or is there some practical use that they have in mind today? Either way, I can see myself spending time implementing techniques and investigating phenomena just like this, even if it were just for the fun of it.

Deep Learning Poised to ‘Blow Up’ Famed Fluid Equations | Quanta Magazine

Deep Learning Poised to ‘Blow Up’ Famed Fluid Equations | Quanta Magazine

How Infinite Series Reveal the Unity of Mathematics:Quanta Magazine

Sometimes the insight that one can get from mathematics can feel like magic. If you allow yourself to follow the logic, you can find constructs in different areas of mathematics that seem to be unrelated to each other that somehow fit with each other in a logical and intellectual way that feels as real as any physical experience – like something you can touch and feel. Perhaps one can find a way to really begin to appreciate how the truths of math and science are foundational to our lives these days. Sadly, there are easier paths to other kinds of “beliefs”, paths that seem to require less effort to follow, which in the end can turn out to be very destructive to our collective existence.

At the least, check out the two puzzles in the Quanta article that I am providing a link for here. Follow your intuition!

https://www.quantamagazine.org/how-infinite-series-reveal-the-unity-of-mathematics-20220124

The Webb Space Telescope Will Rewrite Cosmic History. If It Works.: Quanta Magazine

Collage illustration showing the Webb telescope as a blossoming flower surrounded by stars, galaxies, planets and constellations.

Image from Quanta Magazine

Until I read this article, I did not recognize how much the study of Cosmology and the history of the Universe is still in its infancy. There can be theories about the nature of the Universe, and about how things came to be, but, at the same time, it gets more and more difficult to establish the facts about time closer to the Big Bang through observations. With newer tools, newer discoveries are being made with fair regularity. Many of these discoveries have happened during my lifetime, but I have failed to recognize their significance, for example, the discovery of the existence of exoplanets on other galaxies, the fact that the Universe is not in a steady state and is growing in size, etc… Since I am not an expert in the field, I have the tendency to take things for granted.

If the James Webb Telescope works, we will be able to see earlier into the life of the Universe than ever before. Read the article to understand the details. Energy is still reaching us from earlier on in time from the farthest reaches of the Universe. We just need to be able to detect it with our tools. We still do not have the capability to see things that are close to the Big Bang. Hubble revealed a lot about the early Universe. Hopefully this telescope gets us even closer. This is fascinating stuff.

The article talks about the long developmental history of the telescope, and about its design and the nature of its operation. The challenges include design of this large piece of equipment so that it can be launched into space by rockets of today, a design that can detect extremely low levels of energy from distant galaxies, etc… This is probably the most sophisticated and precise instrument that has ever been built by human hands. Its cost approaches 10 billion dollars, about the cost of building the US Navy’s aircraft carriers today. If even one element of the implementation and deployment goes wrong, we have a problem that would be very difficult to recover from. When deployed about 1 million miles out in space, there is no physical access to the telescope. (That was a lifeline we had with the Hubble telescope.) This is the cost of discovery today.

The first possible launch date for the James Webb telescope is December 22nd of this year. The telescope is in Kourou, French Guiana, at the Ariane launch site. It has been fueled up..

It is a long article, but well worth it, I thought…

https://www.quantamagazine.org/why-nasas-james-webb-space-telescope-matters-so-much-20211203

This is a video from the the article.

Quantum Computers, Explained With Quantum Physics : Quanta Magazine

I am not sure I will ever understand the concepts of Quantum Computing completely, but I still a kick when I get a feeling that I have gotten a somewhat intuitive sense for what is going on. For me to get beyond this state of understanding, I have to put in a far greater level of effort than I am capable of doing at this time. For now, this will suffice.

Federico Ardila on Math, Music and the Space of Possibilities: Quanta Magazine

This is beautiful! Combinatorics, geometry, algebra, multidimensional spaces, robotics, music, education, social awareness – this podcast episode touches on all of this! This guy is a remarkable human being.

https://www.quantamagazine.org/federico-ardila-on-math-music-and-the-space-of-possibilities-20210329/

Life of Bacteria

Did you know that the single-cell bacteria are able to communicate with each other, and that they have the ability to detect the presence, and numbers, of other bacteria around them through a process called “quorum sensing”? Individually, they cannot do much, but when they sense enough of their numbers around them they can behave in a collective manner. The result can be either good or bad for human beings when this collective behavior happens within their bodies. There is strength in numbers! Also, there are more bacteria in the human body than human cells. It also appears that the behavior of bacteria when it comes to organization is very similar to that in human beings. Of course, bacteria existed billions of years before human beings did! You could call the behavior of bacteria intelligence. I blogged about this kind of collective behavior in ants a while back.

https://www.quantamagazine.org/bonnie-bassler-on-talkative-bacteria-and-eavesdropping-viruses-20210308/

Tiktaalik

Nothing, of course, begins at the time you think it did.

I listened to a fascinating podcast a couple of days ago. It had to do with evolution, and the transition on Earth of living forms from fishes (of the water) to creatures who lived on dry land. Perhaps you, like I, have come across some pictures in the media in this regard that try to illustrate the concept in a easy to follow manner. The illustrations could include a body of water on one side and dry land on the other, and show a series of creatures emerging from the water onto the dry land, with the nature of the creatures changing form as you sequence them from the water on to the land. At one end of the sequence you will find a fish. At the other end you will find a human being. Here is an humorous example.

Of course, the pictures do not represent anything close to reality. The transition from fish form to human being took place over hundreds of millions of years and not in single picture frame – obviously. The process was also very complex, and impossible to capture in pictures like this. Also, if I understand correctly, there were simpler forms of life on earth before the fish. Nevertheless…

When scientists study evolution, they try to find evidence of the transitions from one kind of life form to another. This is the realm of the paleontologists. This is a fascinating subject, especially when you are dealing with the study of fossils/skeletons of lifeforms that existed hundreds of millions of years ago. It seems that we know enough about the geology of the earth and the ancient land forms that used to exist in those days, including the mountains, rivers, and oceans, to have some idea as to where to look for pieces of evidence of life from those times. And, surprising to me, there are such land forms, from those times, that are accessible to us easily. For example, there was a section of the Pennsylvania turnpike that was built by blasting a path through a mountainside that revealed rocks over 350 million years old. These rocks revealed preserved fossils from that period of time. (Human beings are capable of destroying our sources of knowledge without even a second thought in our quest for progress and all things “modern”, including mindless and unlimited convenience and speed.)

The reader will surely agree that, as part of the evidence of the evolution that took place, it would be great to find the lifeforms that represent the transition from a form of life that existed solely in the waters to one that lived solely on land, i.e., the fish to tetrapod transition. You may be surprised to learn that the first of this evidence was only discovered in 2004. This life form was given the name Tiktaalik (for reasons you will discover if you follow the links I am providing). The scientific process in this case allowed the scientists to narrow down the time-frame of possible existence of the kind of creature they were looking for, and then look for places where they could access the right kind of rocks of that particular period of time in order to search for the creature. They were successful in their quest.

My blog includes only a small part of the things I learnt from the podcast that I listened to. There is no way I, with my limited understanding, can do justice to the subject matter in a blog. Hopefully, I have stirred your curiosity, and motivated at least one or two of you to also listen to the podcast. Science is fascinating!

https://www.quantamagazine.org/neil-shubin-on-tiktaalik-ballistic-tongues-and-evolution-20210302/