Some animals are ready to go as soon as they are born. These are called precocial animals. They are born knowing how to walk.
It's interesting seeing what comes built-in. You can see this if you watch a horse being born. Within the first hour, the foal will stand, and despite long legs, this usually works the first time. Lying down, however, is not preprogrammed. I've watched a foal circle trying to figure out how to get down from standing, and finally collapsing to the ground in a heap. Standing up quickly is essential to survival, but smoothly lying down is not.
Within a day, a newborn foal can run with the herd.
Of the mammals, most of the equines and some of the rodents (beavers) are precocial.
Pigs are, monkeys are not. It's not closely tied to evolutionary ancestry.
> It's interesting seeing what comes built-in. You can see this if you watch a horse being born.
A fascinating example of this are some Labrador retrievers. Labs are descended from a Newfoundland "landrace" of dogs known as St Johns Water Dogs. They have multiple aquatic adaptations: the "otter tail", oily fur, and webbed feet. (Some of these are shared with other water-oriented breeds.) Some lines of Labradors, especially the "bench" or English dogs, normally retain this full suite of water adaptations.
But the wild thing about these particular Labradors is that they love to swim, and that most of them are born knowing how to swim very well. But they don't know that they know how to swim. So many a young Lab will spend a while standing on the shore, watching humans or other dogs in the water, and fussing because they don't dare to join the fun. Then they may (for example) eventually lean too far and fall into shallow water. Within moments, they'll typically be swimming around and having the time of their lives.
The near-instant transformation from "fascinated by water and fearing it" to "hey I can swim and this is the absolute best thing ever" is remarkable to watch, though not recommended.
I remember another Lab, who'd been afraid to go swimming, who one day impulsively bolted for the water, took an impressive leap off a rock, and (from his reaction) apparently realized in mid-air that he had no idea what he was going to do next. Once he hit the water, he was fortunately fine, to the great relief of his owner.
CAUTION: This behavior pattern is apparently NOT universal in Labs. Owners of "field" or American Labs seem to have much better thought-out protocols for introducing hunting dogs to water, and failure to follow these protocols may result in bad experiences, dogs that fear water, and actual danger to dogs. So please consult an expert.
One of the most curious things I learned about babies is that they are born with a walking instinct, long before they actually can walk. If you hold them up, they will move their legs in a perfectly correct walking fashion. But they lack the strength and agility to keep their body up. At around 3 months this walking instinct disappears, and then at around a year we 'relearn' to walk when we have the strength and agility to hold ourselves up.
But if we were on a planet with significantly lower gravity, humans would likely be walking very near immediately.
If we were on a planet with significantly lower gravity, walking would be much more difficult. Notably, on flat ground we absolutely must have an upward component to our application of force with the surface - this is clearly seen in videos taken on the lunar surface during the Apollo missions. This baby on a hypothetical lower gravity world would find standing easier, yes, but not mobility. At least not once he's taken his first few trail steps.
That series strived for realism in that regard, and in using magnetic boots to work in zero gravity; which was admirable. That made the things that were not realistic stand out even more imo. The (unfortunately named) Epstein drive, a drive that consumes very little mass under constant acceleration allows for relativistic speeds in very little time (weeks). Their ships were flying from one side of the solar system to the other in weeks, but they couldn’t make interstellar flights? Also the effects of cosmic rays and hard radiation on reproduction makes the disaffected belter population seem impossible. That’s all fine of course, just inconsistent imo.
Shohreh Aghdashloo performance was a real treat though!
I did enjoy the first season of the series, but then was turned off by some story arcs, but maybe I will give it a try again. Are the books more consistent?
One of my kids could stand on the day she was born. She seemed super strong, so while I held her I just took my hands away, and she stood there and stared at the rest of the family. Lasted a good 10 seconds, then I thought it was enough.
My boy is 2mo old and he could lock his legs with extreme strength in the first few days. I was very impressed, but my wife told me to stop letting his legs hold any weight. Apparently his uncle was walking at 9mo but his body wasn't ready and he gave himself a hernia.
IIRC Andrej Karpathy in a recent talk made a point that reading a book isn't like memorizing the book, it's more like prompting the brain with the book.
So maybe this concept of being ready to go at birth isn't about the animals ability to start doing things but just a way of upbringing regardless of how ready the animal is to function. Maybe pigs just start prompting early. AFAIK human babies can swim right out of the womb. In other words, maybe the distinction between precocial and non-precocial(I don't know if there's a word for that) animals isn't that clear?
I'm not aware of any way we can know this. We do know that those species are born with the physical ability to walk within the first few hours after birth. How could we distinguish between whether they were born with the knowledge of how to walk as opposed to them learning it quickly since their body can physically do it?
is it true that it's a tradeoff ? the "more precocial" the less flexibility to learn new things ? on the contrary knowing less equals less assumptions, which needs more flexibility in exchange.
Would be true that what is precocial in us is the ability mimic and abstract specific patterns into general rules ?
It must be a tradeoff. I don't have any proof, but my thinking is that we pay an extraordinary price in terms of resources required to keep human babies safe for years before they can keep themselves safe. That is a strong selection pressure on everyone involved. The fact that it still happens means it must somehow be worth it.
I thought you misspelled presocial, but precoial is etymologically related to precocious, both originally meaning early-maturing or something along those lines.
This is unlikely to be a good way to think about them. The norm is for animals to be born knowing how to move. Whether they actually can move shortly after birth is more of a question of muscle development than knowledge.
For example, when birds are held immobile until they're old enough to fly, they fly normally.
There was quite an interesting discussion with Hinton explaining to Jon Stewart how he thinks of brain function, kind of anthropomorphising neurons as things that can see patterns and all they can do is go ping when they see one (https://youtu.be/jrK3PsD3APk?t=366) So if one in the retina say pings if it sees a line and then one behind might see a pattern of a horizontal line ping and a vertical line ping and ping for a cross.
Anyway I was thinking for that to work the neurons would have to kind of chat to each other like "here I am, who's receiving me" etc. Also some communication that if you are differentiating say crosses and circles, the cross neuron can say "hey I've got this one" so the other can go "ok, I'll do the circle then" so the neurons differentiate to recognize different things.
I guess some of that sort of communication system maybe goes on before there is sensory input so the neurons kind of know how they are wired?
No. The headline does not match the justification in the article. The organoid brain tissue is not hooked up to sensory mechanisms in its first months, I accept that, but they are under the influence of input-output training in their initial structure which will would reasonably form some non-random pattern of weights, due to characteristics of the cells.
The question then is, 1) are these characteristics acting as some kind of evolutionary adaption that passes on preconfigured world recognition (asserted by the headline), 2) are they some kind of evolutionary adaptation that makes more effective thinking systems in the form of some specific cognitive structure (more likely IMO), ie they are random features that cause non-random neural structure that drive survival-selection.
Think about the process for (1) to occur. Some ancestor learned in their life to fear snake-like animals or crave mama’s nearness, what possible process could put that knowledge (neural structure of such specificity) into the way that animal generated its sperm or egg? On the other hand, it’s reasonable to assume some genetic encodings encourage specific neural structures even in very early stages, that these are random, but that evolution favored some vs others over the 500mm years animals-with-brains have been around.
I don't know if it's comedy or tragedy, but I've often considered the situation of what a newborn is "expecting".
"Ok, first thing when I come out is I'm gonna meet the family. I'll try to get used to their face, whoever I see first. Maybe they'll show me around the savannah, should be a lot of sunshine, colours, blue sky. Then I'll sleep directly on my mom and get some boob milk."
Kid comes out, everyone is wearing a mask, half of them aren't family, they're indoors with artificial light, and they have "clothes" put on them, and are put in a cot to sleep.
"I need to start use my own lungs to breathe and if I can't do that I'm dead in a minute". Followed by trying to get a milk from a mother who often doesn't yet produce any... Using the eyes to look around is WAY down the list :)
Every little Slopčik was a an even smaller Slopčik, once, whether ageing grandparent, grown parent, teen mom, or the little Slopčik being build to start WW III.
I assume that most parts of the brain that code and decode understanding use argument packing and unpacking, which is a hypothesis I have that builds on another hypothesis that leans on the separation of mind and brain.
There is a level of obfuscation that evolves to be more and more resilient to injection attacks, which is why some children and people almost never fall for bullshit even though they have not been trained for it and have not been exposed to it much more than people who are gullible.
The other side is people who are explicitly trained to be gullible, which happens a lot to and within lower segments of all classes. It was very interesting to observe this in the past few decades and compare it with the notes of previous generations.
The world of the brain is the body, the extension of it can be any environment, as long as the body is build to survive in it. So it is very likely, based on my hypothesis, which I just whipped up out of the blue, that brains would evolve pretty much the same in different environments, with slightly to entirely different bodies around them.
Million of years and dirt knows how many mutations out of even more that would have worked just as well. The brain is doing exactly that, which a lot, if not most, hope AI will never be capable of.
Wow! Some years ago I was thinking about reasons for why people on ADHD/autism spectrum are different.
First heard somewhere (don't remember where or exact idea) that neurons initially form groups and these groups then perform functions. This led to an idea that if someone's brain sacrificed some "copy other primate" groups for "pattern recognition" groups, you would get a unit with higher IQ for non social use, without changing the brain to be more effective in general. This would come at a cost to social/copying skills. This idea doesn't explain "systems thinking" tendency or "not seeing forest for the trees" tendency in autist spectrum folks.
On another occasion, it occurred to me that regular brain run / loop consists of a short reality check and longer flow state. If there are too many reality checks, you get anxiety and can't work effectively. OTOH, too little realty checks and you get stuck on non important things. At the same time, impairing this "check to flow" balance in a safe (non anxiety provoking) environment would result in an individual that could perform the kind of deeper work with results not achievable by not modified individuals.
Have watched 50+ h of psychology lectures, but don't have any formal knowledge on these things so please take it with a grain of salt.
Edit: myself I'm formally on ADHD, and in personal opinion also on Autism spectrum. Just learned to "act normal" very well by the time I got into diagnosis.
> This would come at a cost to social/copying skills. This idea doesn't explain "systems thinking".
The concept of envy/malice/insecurity and people lying to your face and stabbing you in the back was completely foreign to me up to the age of 36.
Only in the face of overwhelming evidence and harm to myself did it all click.
Lately I'm seeing myself in this junior dev I'm mentoring, I'm strongly suspecting he's on the spectrum (that's why he was rejected initially from an internship, despite my input that he'd make a great dev, which proved 100% accurate) -- the guy is totally happy in his technical world, jabs and callous remarks from others completely go over his head.
A lot of people on the spectrum simply have a deep interest in things and systems. I could be wrong, but I think some of those spindle neurons and circuitry made to model others just get used in some people to get systems.
I often get frustrated because people seem to want to learn HOW a technical insight and it's impossible for me to tell them HOW I got to that conclusion, other than I deeply immersed myself in it and it just clicks.
I get the same awe when my wife makes jokes about a behavior of mine or someone else and I can see just how deep, funny and plausible her whole internal model of others is; and sometimes how wrong it is, just like my internal model of a system sometimes is. Alas I can change my internal models of systems on a whim.
Without reading the article, the headline, taken at face value, should come with the caveat that human brain is preconfigured with instructions for understanding the world we've evolved to inhabit. Modern industrial civilization is something different. I wonder to what degree common mental disorders would count as disorders outside the highly unnatural environments and systems we've built for ourselves.
I feel like people on the autism spectrum would still be worse off in a pre-civilization pre-agrarian world, but ADHD would make pretty much no difference.
How newborn brain works is absolutely fascinating for me. I just don't understand how is it possible.
Human DNA contains 1.5 GB information.
Human body, including brain, gets built using this information only. So our "preconfigured" neural networks are also built using this information only.
And apparently it's enough to encode complex behaviour. That's not just visible things. Brain processes a humongous amount of information, it basically supports living processes for entire body, processing miriads of sensors, adjusting all kinds of knobs for body to function properly.
I just don't understand how is it possible just from a purely bit size approach. For me, it's a mystery.
It somewhat makes sense if you think of it in terms of a really complicated 1.5GB metaprogram with a huge pile of conditionals that are triggered by the programs it itself writes (proteins).
The final you is made up of an incomprehensible huge number of copies of the metaprogram, running different configurations, and spitting out programs to each other which then do more stuff.
Our human brains can't really conceive of a configurable metaprogram that writes programs by interacting with itself in different configurations that it itself sets up.
There is a finite size program that can generate infinite digits of pi (in infinite time). Kolmogorov's complexity of pi is finite even when the object is infinite.
It's not very surprising that it takes a few GB of a program to encode conscious 'us'. Humbling to think about it though...
For a demonstration of Kolmogorov complexity, it's good to watch "A mind is born"[0] by lftkryo. It's only 256 bytes, but can generate over 2 minutes of complex music and video. Also, the name is appropriate for this topic :D
It was also developed “iteratively” under extremely harsh selection criteria over a time scale that is so long it is almost impossible to reason about. An old geology textbook I had used the analogy of a geologic timeline that stretched from LA to NYC. Life appears really early (in CA somewhere IIRC) and human existence is about the width of a crack in the pavement just before you hit the Atlantic Ocean.
As a person who knows next to nothing about how the brain or the genes that configure it work, I tend to think of this in terms of 80s video games like River Raid. The level data for these games, if stored naively, would fill the computer's available memory many times over. So they just store a pseudorandom number generator seed along with a few other parameters. Coupled with a few rules to make the level playable, it can generate a seemingly impossible number of levels with very little stored data.
Maybe the genes just encode a few crucial rules and the rest just emerge from that.
Oh, and I know even less about how the universe works. But I tend to think of it in the same terms: Emergent phenomena stemming from simple rules à la Game of Life.
Ever since I read about Rodney Brooks and his idea of the Subsumption architecture I've been convinced that something like this is going on in our minds - likely with some other mechanisms too. It just clicks for me - I'm mostly likely completely wrong, but it's a pretty cool idea, and I've used it to create some really interesting simulations.
Nature recently posted an interesting video [1] about what causes developing hearts to have their first beat. The gist is that eventually random electrical noise triggers a propagating wave which is then continued and repeated by the cellular automation nature of heart tissue. You don't need as much software if your system is composed of emergent properties.
I think it's a wrong way to look at it. In addition to DNA information content, one should count also the complexity of the proteins and higher-level structures in the gametes.
Have you looked into the amazing things people do with procedural generation with only a tiny bit (kilobytes, often) of source code? My intuition is that this is vaguely analogous.
Here's an example from 2003, where the entire source code, from music to visuals, fits in 64 kb: https://youtu.be/HtJvSvQnep0
No they don't. There is plenty of external stimuli that also serves as input, e.g. the process of raising a child and complex thoughts that may only be transferred from grown human to grown human.
Try raising a human in a barren cell without human contact or as part of a pack of wolfs and you'll see how much a human brain is built from "DNA only".
Certainly, and I don't think anyone really doubts this.
Still, people are sometimes surprised by how DNA may affect more parts of behavior than they previously thought.
Not necessarily by directly coding for the behavior. In many cases, the DNA will just modulate how we learn from the environment. And if the environment is fairly constant, observed behavior can correlate more strongly with DNA that one might have expected.
Yeah compared to animals we have a lot of extra bootstrapping outside of physics/chemistry alone via culture and stored information similar to how cell DNA bootsraps via physics, human mind boostraps via stored information in human "network" (talking, internet, books) after being born.
It’s like one computer with program (DNA) and helper programs (RNA) creates second generation of computer and programs (proteins, lipids, carbohydrates) that essentially create their own version of computer system in which they govern things like enzymes, hormones etc
But keep in mind that humans are not created in vacuum. After those two levels of computer create third level that is brain, actual programming of brain is done by other living humans.
So actual „humanity” is what persists in living population and would reset when population is culled and newborn must live and learn on their own.
Even if such newborn would live long enough to have access to things like books, computers, even sound and video records… those would be completely useless to them because they won’t even know language and skills required to use those.
I don't think human DNA generally codes for the behavior derectly. Rather, DNA can code for how the brain learns from incoming data streams.
If the brain naturally tunes into some sources or patterns of input rather than others, it may learn very quickly from the preferred sources. And as long as those sources carry signals that are fairly invariant over time, it may seem like those signals are instinctual.
For instance, it may appear that humans learn to build relationships with kin (both parents and children) and friends, to build revenue streams (or gather food in more primitive societies) and reproduce.
Instead, the brain may come preloaded to generate brain chemicals when detecting certain stimuli. Like oxytocin near caregivers (as children) or small fluffy things (as adults). When exposed to parents/babies, this triggers. But it can also trigger around toys, pets, adopted children, etc.
Friendship-seeking can be, in part, related to seretonin-production in certain social situations. But may be hijacked by social media.
Revenue-seeking behavior can come from dopamin-stimulus from certain goal-optimzing situations. But may also be triggered by video games.
And the best known part: Reproductive behavior may primarily come from sexual arousal, and hijacked by porn or birth control.
Each of the above may be coded by a limited number of bytes of DNA, and it's really the learning algorithm combined with the data stream of natural environments that causes specific behaviors.
A lot of animals are born "hardcoded" with most of the instincts they need to survive, so some behaviors are clearly innate.
And "how the brain learns from the incoming data streams" is, in part, driven a set of behaviors too.
A baby's eyes are trying to detect and track certain preset features long before the primary visual cortex learns to make sense of them. That's a behavior, and it exists for a reason. As the baby develops, the baby would try to seek out certain experiences to learn from them, which is a behavior that exists for a reason too.
There's a hypothesis that certain mental disorders are caused by this innate learning process going off course, but it's just a hypothesis, of the kind that's hard to prove conclusively.
Well, nature has a big advantage over us in that it doesn't need to "make sense" of that code :). So it can, for example, do crazy reusage optimization patterns. A "subroutine" that is used in one place could also be part of a "data piece" of another part. A "header" part can also double down as a "validator" of another part. Doesn't need to make sense, it just needs to work. The only limits are the laws of physics. I would not even call it compression at this point. It's more like heavily optimized spaghetti code.
> Human body, including brain, gets built using [DNA] information only
I think there is a good chance there are other substantial information transfers from one generation to the next. The total genome of all that gut bacteria is orders of magnitude larger for example.
The even crazier thing is that DNA does not encode any of that. Behaviour and morphology is not directly encoded in there, you'll only find recipes for proteins. The zigote will divide into billions of cells that share that same recipe book. Depending on the electric and chemical signals surrounding cells are sending, individual cells get their "personalities" or function. This cell colony forms an organism which emerges from the sum of morphology and behaviour of all cells. But you'll find no recipe for an arm in DNA, it is the result of the work of the collective intelligence that is your body.
I'm not sure in what sense there isn't a recipe for arms in our DNA. To me, it seems the DNA does encode that stuff, but in a highly compressed format that is then "unzipped" through the laws of physics and biology into a living and breathing being with arms.
I mean, the information has to be in there somewhere, right?
Well technically yeah but consider that it takes ~9 months for the product to function without constant life support, at least a few years until majority of the basic functions work and ~15 years until it is fully functional.
9 months is caused by head size to how far you can stretch the exit ratio. In a way, we are born prematurely, to lessen the probability of death in childbirth (for both the mother and child).
It’s not foolproof, but I can easily transmit a huge amount of information to someone by saying “Titanic prow king of the world scene.” In seven words, which could be fewer if I were really trying, the recipient has a moving image and sound in their head (as long as they’re the right age group- every example I could think of made me feel old).
Well that was transmitting "a pointer" more than anything else, but yes I agree that nature could be doing the same thing. Not hardcode behaviors, but certain chemical reactions to some "pointers" that are totally environment related. Arachnophobia apparently could have a genetic component, so there could be a "spider pointer" somewhere.
If compressed, then there is room for more. (but afaik much is rather unused)
And for me I cannot say, that life is not a mystery to me, but this specific part I have less trouble with imagining it. As little code can create complex worlds and simulate them. (a minecraft wasm build for example is just 14 mb, but fully working)
I think you're underestimating the role epigenetic information plays. 1.5 GiB encodes every protein used to build us, sure, but which genes get switched on when and how are sensitive to factors not encoded for in DNA, including the environment of the cell and the fundamental chemistry of biology. Epigenetic information is hard to capture but can profoundly affect how an organism develops; cloned cats, for instance, may show a vastly different fur color and pattern from the original, to cite just a highly visible example.
For me, it's one of the last true mysteries! We've figured out damned near everything else, nothing has this level of "unknown" to it.
It's simply mind-blowing to me how such a tiny block of data can encode such high-level behaviours so indirectly!
Genes code for proteins, not synapse weights!
Those proteins influence cell division, specialisation, and growth through a complex interplay of thousands of distinct signal chemicals.
Then those cells assemble into a brain, apparently "randomly" with only crude, coarse patterns that are at best statistical in nature. Some cells are longer, some shorter, some with more interconnects, some with less, but no two perfectly alike.
Then, then, somehow... waves hands... magically this encodes that "wide hips are sexually attractive" in a way that turns up fully a decade later, well into the "pre-training" phase!!!
What... the... %#%@!
How does that work!? How does any of that work?
Y'all work in AI, ML, or adjacent to it. You know how hard it is to train a model to learn to detect anything even with thousands of examples!
PS: Human DNA contains only 750 MB (62 billion bits) of information, of which maybe 0.1% to 1% directly code for brain structure and the like. Let's be generous and say 10%. That is just 75 MB that somehow makes us scared of snakes and spiders, afraid of heights, attracted to the opposite sex, capable of speech, enjoy dancing, understand on instinct what is a "bad" or "good" smell, etc, etc...
For us it's hard to train a model because our compute and resources is nothing compared to nature's "compute" the whole universe: "it" has absurdly more resources to run different variations and massively parallel compute to run the evolutionary "algorithm", if you think about all the chemical building blocks, proteins, cells, that was "tried" and didn't survive.
From that angle our artificial models seem very sample efficient, but it's all hard to quantify it without know what was "tried" by the universe to reach the current state.
But it's all weird to think about because there is no intent in natures optimizations it's just happens because it can and there is enough energy and parallel randomness to eventually happen.
And the real mystery is not how evolution achieved this but that the laws of chemistry/universe allow self-replicating structures to appear at all. In an universe with different rules it couldn't happen even with infinite trial and error compute.
The brain absolutely and biology in general when one starts digging.
Discovery of DNA was positioned as a "Biology: Mission Accomplished" - it's far from true. We don't understand all of DNA and epigenetics. We don't have a good understanding of how life began.
Back to the brain, it's power consumption to capabilities, weight to capabilities is just insane. The link to brain size and intelligence is a mystery as well - jumping spiders, octopus, corvids, parrots ...
There is still a big discussion of nature vs nurture. Did not follow the subject you mention but many things can be in fact just learned.
Also, as mentioned previously, there is more than the DNA at work - like at least epigenetics, but I guess the fetus is influenced a lot by the mother's body.
Epigenetics and mother's body influence feel - to me - like magic more or less the same. And the nature vs nurture regarding tastes developed either early or later on, well, as a father of 2 siblings who are radically different in certain tastes, I don't really know where I would have nurtured them into being different. I try to introspect a lot on that, maybe we did something but honestly... I don't think so.
I do agree that some organisms will transmit more "information" (via multiple ways, chemically, mechanical, etc.) than others (like maybe the birds) but the fact is the DNA is just a part of the development process and even if maybe it is "the first one", it will not "pack" everything.
This is why I’m so insistent that LLMs aren’t the best way (if they are a way at all) to getting to human level intelligence. The maximum amount of energy and input data required for training and inference is many orders of magnitude less than we are currently using.
I've always found Retinal Waves[1] interesting. During development of the visual system, there are spontaneous bursts of activity without external stimuli, helping the synapses to organize properly.
In my layman's view, it's like hallucinating shapes that are important to learn. Very similar to the "priming" described in the article, but easier to visualize (literally).
AFAIK the whole morphology is decided in distributed computation fashion via electrical potential changes, at least according to experiments by Michael Levin
Maybe I’m overlooking something, but wouldn’t this be similar to an instinct that is preprogrammed from natural selection? For example, sea turtles know they need to move from the beach toward the ocean, and spiders know how to spin their species specific web pattern. No-one teaches the sea turtles or spiders how to do this. Wouldn't this be the same for our thoughts and thinking?
This seems like it was proven ages ago with the no-free-lunch theorem.
Humans could not learn to function unless their brains encode a useful prior for learning about the world. That prior means "preconfigured with instructions for understanding the world".
The short form of the no-free-lunch theorem is that if there is no prior (i.e. all possible universes are equally likely) then for any learning problem P there are an equal number of universes that learning system A will outdo learning system B on that problem.
If not all universes are equally likely, one learning system can vastly outdo another or even most other learning systems. Not equally likely is the assumption built into brains. Without that, you can't learn effectively.
So the biology is just implementation of that general principle. The details of how that implementation works are interesting, but whether we are preconfigured for learning was never in question.
And we've proven empirically that this usable "prior" can be quite small.
What exact assumptions does human brain encode and how does it use them, however, is an area of research. We are nowhere near being able to list out all of those useful inductive biases - let alone extract them and apply them to AIs.
IMO TFA doesn't map too cleanly onto the concept of "bootloader" (nor "microcode" for that matter). But I guess the question is, as always, can you unlock it.
The range of computational processes a human brain could perform is quite large. The range of computational processes that resemble the behavior of a sane human? Far less so.
There’s a difference between reacting to your environment, and being intelligent. Many top scientists working in AI, I know from firsthand experience, are disconnected from how any form of responsiveness appears in nature. It’s asinine to try to build or recreate something you don’t understand, why even bother?
I never agreed with his views on syntaxis, but the (his?) idea that large parts of our language capabilities are innate is almost beyond doubt. Are people still arguing against it?
I think it's about the details. Chomsky argued a lot of grammar must be innate but the ability of LLMs to do grammar quite well with only a basic artificial neural network argues against that.
One could ask the same question about any trait of any organism ... and the answer is always the same. Do you have a problem with birds being able to build nests specific to their species, or cuckoo chicks instinctively pushing the eggs of the host species out of the nest? The answer is one of the best understood facts of science, and the basis of all of biology. Why would anyone expect the human brain not to be "preconfigured" by the billions of years of environmental forces that produced it?
It's not "chosen". It is evolution. Your DNA has the metaprogram that sets up all the programs in your brain. Most of them are learning programs but you also have hardcoded programs on how to perform your bodily functions, how and when to cry, and how to suck on a tit.
Anthropomorphic language about evolution is simply a convenient metaphor that eases communication ... it has no metaphysical implications.
> You guys have entered the domain of philosophy a long time ago and didn't realize it, thinking it is still empirical science.
This ad hominem sweeping generalization about people you know nothing about is so casually expressed while being so extraordinarily arrogant. Among other fallacies packed into it is a radically false dichotomy.
Some animals are ready to go as soon as they are born. These are called precocial animals. They are born knowing how to walk.
It's interesting seeing what comes built-in. You can see this if you watch a horse being born. Within the first hour, the foal will stand, and despite long legs, this usually works the first time. Lying down, however, is not preprogrammed. I've watched a foal circle trying to figure out how to get down from standing, and finally collapsing to the ground in a heap. Standing up quickly is essential to survival, but smoothly lying down is not. Within a day, a newborn foal can run with the herd.
Of the mammals, most of the equines and some of the rodents (beavers) are precocial. Pigs are, monkeys are not. It's not closely tied to evolutionary ancestry.
> It's interesting seeing what comes built-in. You can see this if you watch a horse being born.
A fascinating example of this are some Labrador retrievers. Labs are descended from a Newfoundland "landrace" of dogs known as St Johns Water Dogs. They have multiple aquatic adaptations: the "otter tail", oily fur, and webbed feet. (Some of these are shared with other water-oriented breeds.) Some lines of Labradors, especially the "bench" or English dogs, normally retain this full suite of water adaptations.
But the wild thing about these particular Labradors is that they love to swim, and that most of them are born knowing how to swim very well. But they don't know that they know how to swim. So many a young Lab will spend a while standing on the shore, watching humans or other dogs in the water, and fussing because they don't dare to join the fun. Then they may (for example) eventually lean too far and fall into shallow water. Within moments, they'll typically be swimming around and having the time of their lives.
The near-instant transformation from "fascinated by water and fearing it" to "hey I can swim and this is the absolute best thing ever" is remarkable to watch, though not recommended.
I remember another Lab, who'd been afraid to go swimming, who one day impulsively bolted for the water, took an impressive leap off a rock, and (from his reaction) apparently realized in mid-air that he had no idea what he was going to do next. Once he hit the water, he was fortunately fine, to the great relief of his owner.
CAUTION: This behavior pattern is apparently NOT universal in Labs. Owners of "field" or American Labs seem to have much better thought-out protocols for introducing hunting dogs to water, and failure to follow these protocols may result in bad experiences, dogs that fear water, and actual danger to dogs. So please consult an expert.
One of the most curious things I learned about babies is that they are born with a walking instinct, long before they actually can walk. If you hold them up, they will move their legs in a perfectly correct walking fashion. But they lack the strength and agility to keep their body up. At around 3 months this walking instinct disappears, and then at around a year we 'relearn' to walk when we have the strength and agility to hold ourselves up.
But if we were on a planet with significantly lower gravity, humans would likely be walking very near immediately.
If we were on a planet with significantly lower gravity, walking would be much more difficult. Notably, on flat ground we absolutely must have an upward component to our application of force with the surface - this is clearly seen in videos taken on the lunar surface during the Apollo missions. This baby on a hypothetical lower gravity world would find standing easier, yes, but not mobility. At least not once he's taken his first few trail steps.
The moon has very little gravity bringing extra problems, but maybe Mars would have the right gravity to enable Babies walk from the beginning?
If you enjoy this kind of speculating you might like the Expanse series of books and TV shows.
They have humans growing up on Mars, the asteroid belt, moons. Anyone who doesn’t grow up on earth cannot go there without extreme gravity training.
That series strived for realism in that regard, and in using magnetic boots to work in zero gravity; which was admirable. That made the things that were not realistic stand out even more imo. The (unfortunately named) Epstein drive, a drive that consumes very little mass under constant acceleration allows for relativistic speeds in very little time (weeks). Their ships were flying from one side of the solar system to the other in weeks, but they couldn’t make interstellar flights? Also the effects of cosmic rays and hard radiation on reproduction makes the disaffected belter population seem impossible. That’s all fine of course, just inconsistent imo.
Shohreh Aghdashloo performance was a real treat though!
I did enjoy the first season of the series, but then was turned off by some story arcs, but maybe I will give it a try again. Are the books more consistent?
Much more consistent. Books are huge, hence the need to shorten them.
But IMHO, series have done a really good job overall. Given how nearly impossible it is to simulate micro-gravity, or other advanced technology.
Also, in season three or four suddenly everybody started cursing all the time. The series just wasn't fun to watch anymore.
I legitimately did not notice this and I cannot imagine it affecting my enjoyment of a show.
Walking would probably suck on such a planet and we would see babies bounding long distances instead!
One of my kids could stand on the day she was born. She seemed super strong, so while I held her I just took my hands away, and she stood there and stared at the rest of the family. Lasted a good 10 seconds, then I thought it was enough.
My boy is 2mo old and he could lock his legs with extreme strength in the first few days. I was very impressed, but my wife told me to stop letting his legs hold any weight. Apparently his uncle was walking at 9mo but his body wasn't ready and he gave himself a hernia.
IIRC they also have a swimming/diving instinct/reflex, which they similarly seem to unlearn after a while.
IIRC Andrej Karpathy in a recent talk made a point that reading a book isn't like memorizing the book, it's more like prompting the brain with the book.
So maybe this concept of being ready to go at birth isn't about the animals ability to start doing things but just a way of upbringing regardless of how ready the animal is to function. Maybe pigs just start prompting early. AFAIK human babies can swim right out of the womb. In other words, maybe the distinction between precocial and non-precocial(I don't know if there's a word for that) animals isn't that clear?
I don't think babies can swim but they know not to try and breathe in water. Which is probably what you meant.
> They are born knowing how to walk.
I'm not aware of any way we can know this. We do know that those species are born with the physical ability to walk within the first few hours after birth. How could we distinguish between whether they were born with the knowledge of how to walk as opposed to them learning it quickly since their body can physically do it?
How about running from snakes for their lives right after they hatch: https://www.youtube.com/watch?v=el4CQj-TCbA
https://en.wikipedia.org/wiki/Central_pattern_generator#Loco...
> Some animals are ready to go as soon as they are born
That trade has an extreme genetic advantage when other animals see you as their succulent mains on the a la carte exotic wildlife menu.
is it true that it's a tradeoff ? the "more precocial" the less flexibility to learn new things ? on the contrary knowing less equals less assumptions, which needs more flexibility in exchange.
Would be true that what is precocial in us is the ability mimic and abstract specific patterns into general rules ?
It must be a tradeoff. I don't have any proof, but my thinking is that we pay an extraordinary price in terms of resources required to keep human babies safe for years before they can keep themselves safe. That is a strong selection pressure on everyone involved. The fact that it still happens means it must somehow be worth it.
Humans are born quite prematurely so that the head fits through the birth canal.
> precocial
I thought you misspelled presocial, but precoial is etymologically related to precocious, both originally meaning early-maturing or something along those lines.
> They are born knowing how to walk.
This is unlikely to be a good way to think about them. The norm is for animals to be born knowing how to move. Whether they actually can move shortly after birth is more of a question of muscle development than knowledge.
For example, when birds are held immobile until they're old enough to fly, they fly normally.
There was quite an interesting discussion with Hinton explaining to Jon Stewart how he thinks of brain function, kind of anthropomorphising neurons as things that can see patterns and all they can do is go ping when they see one (https://youtu.be/jrK3PsD3APk?t=366) So if one in the retina say pings if it sees a line and then one behind might see a pattern of a horizontal line ping and a vertical line ping and ping for a cross.
Anyway I was thinking for that to work the neurons would have to kind of chat to each other like "here I am, who's receiving me" etc. Also some communication that if you are differentiating say crosses and circles, the cross neuron can say "hey I've got this one" so the other can go "ok, I'll do the circle then" so the neurons differentiate to recognize different things.
I guess some of that sort of communication system maybe goes on before there is sensory input so the neurons kind of know how they are wired?
No. The headline does not match the justification in the article. The organoid brain tissue is not hooked up to sensory mechanisms in its first months, I accept that, but they are under the influence of input-output training in their initial structure which will would reasonably form some non-random pattern of weights, due to characteristics of the cells.
The question then is, 1) are these characteristics acting as some kind of evolutionary adaption that passes on preconfigured world recognition (asserted by the headline), 2) are they some kind of evolutionary adaptation that makes more effective thinking systems in the form of some specific cognitive structure (more likely IMO), ie they are random features that cause non-random neural structure that drive survival-selection.
Think about the process for (1) to occur. Some ancestor learned in their life to fear snake-like animals or crave mama’s nearness, what possible process could put that knowledge (neural structure of such specificity) into the way that animal generated its sperm or egg? On the other hand, it’s reasonable to assume some genetic encodings encourage specific neural structures even in very early stages, that these are random, but that evolution favored some vs others over the 500mm years animals-with-brains have been around.
I don't know if it's comedy or tragedy, but I've often considered the situation of what a newborn is "expecting".
"Ok, first thing when I come out is I'm gonna meet the family. I'll try to get used to their face, whoever I see first. Maybe they'll show me around the savannah, should be a lot of sunshine, colours, blue sky. Then I'll sleep directly on my mom and get some boob milk."
Kid comes out, everyone is wearing a mask, half of them aren't family, they're indoors with artificial light, and they have "clothes" put on them, and are put in a cot to sleep.
It's A LOT more stressful than that :)
"I need to start use my own lungs to breathe and if I can't do that I'm dead in a minute". Followed by trying to get a milk from a mother who often doesn't yet produce any... Using the eyes to look around is WAY down the list :)
Every little Slopčik was a an even smaller Slopčik, once, whether ageing grandparent, grown parent, teen mom, or the little Slopčik being build to start WW III.
I assume that most parts of the brain that code and decode understanding use argument packing and unpacking, which is a hypothesis I have that builds on another hypothesis that leans on the separation of mind and brain.
There is a level of obfuscation that evolves to be more and more resilient to injection attacks, which is why some children and people almost never fall for bullshit even though they have not been trained for it and have not been exposed to it much more than people who are gullible.
The other side is people who are explicitly trained to be gullible, which happens a lot to and within lower segments of all classes. It was very interesting to observe this in the past few decades and compare it with the notes of previous generations.
The world of the brain is the body, the extension of it can be any environment, as long as the body is build to survive in it. So it is very likely, based on my hypothesis, which I just whipped up out of the blue, that brains would evolve pretty much the same in different environments, with slightly to entirely different bodies around them.
Million of years and dirt knows how many mutations out of even more that would have worked just as well. The brain is doing exactly that, which a lot, if not most, hope AI will never be capable of.
Wow! Some years ago I was thinking about reasons for why people on ADHD/autism spectrum are different.
First heard somewhere (don't remember where or exact idea) that neurons initially form groups and these groups then perform functions. This led to an idea that if someone's brain sacrificed some "copy other primate" groups for "pattern recognition" groups, you would get a unit with higher IQ for non social use, without changing the brain to be more effective in general. This would come at a cost to social/copying skills. This idea doesn't explain "systems thinking" tendency or "not seeing forest for the trees" tendency in autist spectrum folks.
On another occasion, it occurred to me that regular brain run / loop consists of a short reality check and longer flow state. If there are too many reality checks, you get anxiety and can't work effectively. OTOH, too little realty checks and you get stuck on non important things. At the same time, impairing this "check to flow" balance in a safe (non anxiety provoking) environment would result in an individual that could perform the kind of deeper work with results not achievable by not modified individuals.
Have watched 50+ h of psychology lectures, but don't have any formal knowledge on these things so please take it with a grain of salt.
Edit: myself I'm formally on ADHD, and in personal opinion also on Autism spectrum. Just learned to "act normal" very well by the time I got into diagnosis.
> This would come at a cost to social/copying skills. This idea doesn't explain "systems thinking".
The concept of envy/malice/insecurity and people lying to your face and stabbing you in the back was completely foreign to me up to the age of 36. Only in the face of overwhelming evidence and harm to myself did it all click.
Lately I'm seeing myself in this junior dev I'm mentoring, I'm strongly suspecting he's on the spectrum (that's why he was rejected initially from an internship, despite my input that he'd make a great dev, which proved 100% accurate) -- the guy is totally happy in his technical world, jabs and callous remarks from others completely go over his head.
A lot of people on the spectrum simply have a deep interest in things and systems. I could be wrong, but I think some of those spindle neurons and circuitry made to model others just get used in some people to get systems.
I often get frustrated because people seem to want to learn HOW a technical insight and it's impossible for me to tell them HOW I got to that conclusion, other than I deeply immersed myself in it and it just clicks. I get the same awe when my wife makes jokes about a behavior of mine or someone else and I can see just how deep, funny and plausible her whole internal model of others is; and sometimes how wrong it is, just like my internal model of a system sometimes is. Alas I can change my internal models of systems on a whim.
Without reading the article, the headline, taken at face value, should come with the caveat that human brain is preconfigured with instructions for understanding the world we've evolved to inhabit. Modern industrial civilization is something different. I wonder to what degree common mental disorders would count as disorders outside the highly unnatural environments and systems we've built for ourselves.
I feel like people on the autism spectrum would still be worse off in a pre-civilization pre-agrarian world, but ADHD would make pretty much no difference.
How newborn brain works is absolutely fascinating for me. I just don't understand how is it possible.
Human DNA contains 1.5 GB information.
Human body, including brain, gets built using this information only. So our "preconfigured" neural networks are also built using this information only.
And apparently it's enough to encode complex behaviour. That's not just visible things. Brain processes a humongous amount of information, it basically supports living processes for entire body, processing miriads of sensors, adjusting all kinds of knobs for body to function properly.
I just don't understand how is it possible just from a purely bit size approach. For me, it's a mystery.
It somewhat makes sense if you think of it in terms of a really complicated 1.5GB metaprogram with a huge pile of conditionals that are triggered by the programs it itself writes (proteins). The final you is made up of an incomprehensible huge number of copies of the metaprogram, running different configurations, and spitting out programs to each other which then do more stuff. Our human brains can't really conceive of a configurable metaprogram that writes programs by interacting with itself in different configurations that it itself sets up.
Something similar: Kolmogorov complexity.
There is a finite size program that can generate infinite digits of pi (in infinite time). Kolmogorov's complexity of pi is finite even when the object is infinite.
It's not very surprising that it takes a few GB of a program to encode conscious 'us'. Humbling to think about it though...
For a demonstration of Kolmogorov complexity, it's good to watch "A mind is born"[0] by lftkryo. It's only 256 bytes, but can generate over 2 minutes of complex music and video. Also, the name is appropriate for this topic :D
[0] https://www.youtube.com/watch?v=sWblpsLZ-O8
It was also developed “iteratively” under extremely harsh selection criteria over a time scale that is so long it is almost impossible to reason about. An old geology textbook I had used the analogy of a geologic timeline that stretched from LA to NYC. Life appears really early (in CA somewhere IIRC) and human existence is about the width of a crack in the pavement just before you hit the Atlantic Ocean.
As a person who knows next to nothing about how the brain or the genes that configure it work, I tend to think of this in terms of 80s video games like River Raid. The level data for these games, if stored naively, would fill the computer's available memory many times over. So they just store a pseudorandom number generator seed along with a few other parameters. Coupled with a few rules to make the level playable, it can generate a seemingly impossible number of levels with very little stored data.
Maybe the genes just encode a few crucial rules and the rest just emerge from that.
Oh, and I know even less about how the universe works. But I tend to think of it in the same terms: Emergent phenomena stemming from simple rules à la Game of Life.
Ever since I read about Rodney Brooks and his idea of the Subsumption architecture I've been convinced that something like this is going on in our minds - likely with some other mechanisms too. It just clicks for me - I'm mostly likely completely wrong, but it's a pretty cool idea, and I've used it to create some really interesting simulations.
https://en.wikipedia.org/wiki/Subsumption_architecture
Nature recently posted an interesting video [1] about what causes developing hearts to have their first beat. The gist is that eventually random electrical noise triggers a propagating wave which is then continued and repeated by the cellular automation nature of heart tissue. You don't need as much software if your system is composed of emergent properties.
[1]: https://youtu.be/SIMS2h5QsZU
I think it's a wrong way to look at it. In addition to DNA information content, one should count also the complexity of the proteins and higher-level structures in the gametes.
Have you looked into the amazing things people do with procedural generation with only a tiny bit (kilobytes, often) of source code? My intuition is that this is vaguely analogous.
Here's an example from 2003, where the entire source code, from music to visuals, fits in 64 kb: https://youtu.be/HtJvSvQnep0
Here's a good gallery of such demos: https://64k-scene.github.io
> gets built using this information only
No they don't. There is plenty of external stimuli that also serves as input, e.g. the process of raising a child and complex thoughts that may only be transferred from grown human to grown human.
Try raising a human in a barren cell without human contact or as part of a pack of wolfs and you'll see how much a human brain is built from "DNA only".
Certainly, and I don't think anyone really doubts this.
Still, people are sometimes surprised by how DNA may affect more parts of behavior than they previously thought.
Not necessarily by directly coding for the behavior. In many cases, the DNA will just modulate how we learn from the environment. And if the environment is fairly constant, observed behavior can correlate more strongly with DNA that one might have expected.
Yeah compared to animals we have a lot of extra bootstrapping outside of physics/chemistry alone via culture and stored information similar to how cell DNA bootsraps via physics, human mind boostraps via stored information in human "network" (talking, internet, books) after being born.
[dead]
> I just don't understand how is it possible just from a purely bit size approach.
It's ~750MB (3 billion base pairs). But anyway, that's a size of a decent Linux distribution with tons of software.
It’s like one computer with program (DNA) and helper programs (RNA) creates second generation of computer and programs (proteins, lipids, carbohydrates) that essentially create their own version of computer system in which they govern things like enzymes, hormones etc
But keep in mind that humans are not created in vacuum. After those two levels of computer create third level that is brain, actual programming of brain is done by other living humans.
So actual „humanity” is what persists in living population and would reset when population is culled and newborn must live and learn on their own.
Even if such newborn would live long enough to have access to things like books, computers, even sound and video records… those would be completely useless to them because they won’t even know language and skills required to use those.
I don't think human DNA generally codes for the behavior derectly. Rather, DNA can code for how the brain learns from incoming data streams.
If the brain naturally tunes into some sources or patterns of input rather than others, it may learn very quickly from the preferred sources. And as long as those sources carry signals that are fairly invariant over time, it may seem like those signals are instinctual.
For instance, it may appear that humans learn to build relationships with kin (both parents and children) and friends, to build revenue streams (or gather food in more primitive societies) and reproduce.
Instead, the brain may come preloaded to generate brain chemicals when detecting certain stimuli. Like oxytocin near caregivers (as children) or small fluffy things (as adults). When exposed to parents/babies, this triggers. But it can also trigger around toys, pets, adopted children, etc.
Friendship-seeking can be, in part, related to seretonin-production in certain social situations. But may be hijacked by social media.
Revenue-seeking behavior can come from dopamin-stimulus from certain goal-optimzing situations. But may also be triggered by video games.
And the best known part: Reproductive behavior may primarily come from sexual arousal, and hijacked by porn or birth control.
Each of the above may be coded by a limited number of bytes of DNA, and it's really the learning algorithm combined with the data stream of natural environments that causes specific behaviors.
A lot of animals are born "hardcoded" with most of the instincts they need to survive, so some behaviors are clearly innate.
And "how the brain learns from the incoming data streams" is, in part, driven a set of behaviors too.
A baby's eyes are trying to detect and track certain preset features long before the primary visual cortex learns to make sense of them. That's a behavior, and it exists for a reason. As the baby develops, the baby would try to seek out certain experiences to learn from them, which is a behavior that exists for a reason too.
There's a hypothesis that certain mental disorders are caused by this innate learning process going off course, but it's just a hypothesis, of the kind that's hard to prove conclusively.
Actually the DNA is very inefficient with many areas that appear to do nothing. 1.5GB is a ton of "source code".
There is no significant evolutionary pressure to erase unnecessary parts.
> 1.5 GB information
Well, nature has a big advantage over us in that it doesn't need to "make sense" of that code :). So it can, for example, do crazy reusage optimization patterns. A "subroutine" that is used in one place could also be part of a "data piece" of another part. A "header" part can also double down as a "validator" of another part. Doesn't need to make sense, it just needs to work. The only limits are the laws of physics. I would not even call it compression at this point. It's more like heavily optimized spaghetti code.
> Human body, including brain, gets built using [DNA] information only
I think there is a good chance there are other substantial information transfers from one generation to the next. The total genome of all that gut bacteria is orders of magnitude larger for example.
The even crazier thing is that DNA does not encode any of that. Behaviour and morphology is not directly encoded in there, you'll only find recipes for proteins. The zigote will divide into billions of cells that share that same recipe book. Depending on the electric and chemical signals surrounding cells are sending, individual cells get their "personalities" or function. This cell colony forms an organism which emerges from the sum of morphology and behaviour of all cells. But you'll find no recipe for an arm in DNA, it is the result of the work of the collective intelligence that is your body.
I'm not sure in what sense there isn't a recipe for arms in our DNA. To me, it seems the DNA does encode that stuff, but in a highly compressed format that is then "unzipped" through the laws of physics and biology into a living and breathing being with arms.
I mean, the information has to be in there somewhere, right?
I don't know either, maybe epigenetics play a part in this (Some information transferred from the mother cells to the child)?
Well technically yeah but consider that it takes ~9 months for the product to function without constant life support, at least a few years until majority of the basic functions work and ~15 years until it is fully functional.
Talk about compile time.
9 months is caused by head size to how far you can stretch the exit ratio. In a way, we are born prematurely, to lessen the probability of death in childbirth (for both the mother and child).
The longer I think about it, the worse it gets.
It’s not foolproof, but I can easily transmit a huge amount of information to someone by saying “Titanic prow king of the world scene.” In seven words, which could be fewer if I were really trying, the recipient has a moving image and sound in their head (as long as they’re the right age group- every example I could think of made me feel old).
> Titanic prow king of the world scene
Well that was transmitting "a pointer" more than anything else, but yes I agree that nature could be doing the same thing. Not hardcode behaviors, but certain chemical reactions to some "pointers" that are totally environment related. Arachnophobia apparently could have a genetic component, so there could be a "spider pointer" somewhere.
"Human DNA contains 1.5 GB information"
If compressed, then there is room for more. (but afaik much is rather unused)
And for me I cannot say, that life is not a mystery to me, but this specific part I have less trouble with imagining it. As little code can create complex worlds and simulate them. (a minecraft wasm build for example is just 14 mb, but fully working)
I think you're underestimating the role epigenetic information plays. 1.5 GiB encodes every protein used to build us, sure, but which genes get switched on when and how are sensitive to factors not encoded for in DNA, including the environment of the cell and the fundamental chemistry of biology. Epigenetic information is hard to capture but can profoundly affect how an organism develops; cloned cats, for instance, may show a vastly different fur color and pattern from the original, to cite just a highly visible example.
Makes one curious about epigenetics!
> For me, it's a mystery.
For me, it's one of the last true mysteries! We've figured out damned near everything else, nothing has this level of "unknown" to it.
It's simply mind-blowing to me how such a tiny block of data can encode such high-level behaviours so indirectly!
Genes code for proteins, not synapse weights!
Those proteins influence cell division, specialisation, and growth through a complex interplay of thousands of distinct signal chemicals.
Then those cells assemble into a brain, apparently "randomly" with only crude, coarse patterns that are at best statistical in nature. Some cells are longer, some shorter, some with more interconnects, some with less, but no two perfectly alike.
Then, then, somehow... waves hands... magically this encodes that "wide hips are sexually attractive" in a way that turns up fully a decade later, well into the "pre-training" phase!!!
What... the... %#%@!
How does that work!? How does any of that work?
Y'all work in AI, ML, or adjacent to it. You know how hard it is to train a model to learn to detect anything even with thousands of examples!
PS: Human DNA contains only 750 MB (62 billion bits) of information, of which maybe 0.1% to 1% directly code for brain structure and the like. Let's be generous and say 10%. That is just 75 MB that somehow makes us scared of snakes and spiders, afraid of heights, attracted to the opposite sex, capable of speech, enjoy dancing, understand on instinct what is a "bad" or "good" smell, etc, etc...
For us it's hard to train a model because our compute and resources is nothing compared to nature's "compute" the whole universe: "it" has absurdly more resources to run different variations and massively parallel compute to run the evolutionary "algorithm", if you think about all the chemical building blocks, proteins, cells, that was "tried" and didn't survive.
From that angle our artificial models seem very sample efficient, but it's all hard to quantify it without know what was "tried" by the universe to reach the current state. But it's all weird to think about because there is no intent in natures optimizations it's just happens because it can and there is enough energy and parallel randomness to eventually happen.
And the real mystery is not how evolution achieved this but that the laws of chemistry/universe allow self-replicating structures to appear at all. In an universe with different rules it couldn't happen even with infinite trial and error compute.
The brain absolutely and biology in general when one starts digging.
Discovery of DNA was positioned as a "Biology: Mission Accomplished" - it's far from true. We don't understand all of DNA and epigenetics. We don't have a good understanding of how life began.
Back to the brain, it's power consumption to capabilities, weight to capabilities is just insane. The link to brain size and intelligence is a mystery as well - jumping spiders, octopus, corvids, parrots ...
There is still a big discussion of nature vs nurture. Did not follow the subject you mention but many things can be in fact just learned.
Also, as mentioned previously, there is more than the DNA at work - like at least epigenetics, but I guess the fetus is influenced a lot by the mother's body.
Epigenetics and mother's body influence feel - to me - like magic more or less the same. And the nature vs nurture regarding tastes developed either early or later on, well, as a father of 2 siblings who are radically different in certain tastes, I don't really know where I would have nurtured them into being different. I try to introspect a lot on that, maybe we did something but honestly... I don't think so.
With humans, we can even imagine that mother body teaches child brains via placenta or something (I don't think that's what happening, but whatever).
However think about birds. They lay eggs. So there's no direct connection between mother body and child body. Yet it works somehow...
The yolk (used directly in the embryos development) is generated during 10 days (https://www.purinamills.com/chicken-feed/education/detail/ho...). This could give the opportunity to pack a lot of "indirect information" to be used by the future embryo.
Regarding "teaching" the child while in the womb, it is exactly what is happening, see: https://www.americanscientist.org/article/baby-talk
I do agree that some organisms will transmit more "information" (via multiple ways, chemically, mechanical, etc.) than others (like maybe the birds) but the fact is the DNA is just a part of the development process and even if maybe it is "the first one", it will not "pack" everything.
Recursion
You can make a brainfuck runtime in less than a kilobyte and it can run any program known to man.
This is why I’m so insistent that LLMs aren’t the best way (if they are a way at all) to getting to human level intelligence. The maximum amount of energy and input data required for training and inference is many orders of magnitude less than we are currently using.
I've always found Retinal Waves[1] interesting. During development of the visual system, there are spontaneous bursts of activity without external stimuli, helping the synapses to organize properly.
In my layman's view, it's like hallucinating shapes that are important to learn. Very similar to the "priming" described in the article, but easier to visualize (literally).
[1] https://en.wikipedia.org/wiki/Retinal_waves
AFAIK the whole morphology is decided in distributed computation fashion via electrical potential changes, at least according to experiments by Michael Levin
Sounds a lot like someone read Kant and said "Let's check how categories could work!"
Maybe I’m overlooking something, but wouldn’t this be similar to an instinct that is preprogrammed from natural selection? For example, sea turtles know they need to move from the beach toward the ocean, and spiders know how to spin their species specific web pattern. No-one teaches the sea turtles or spiders how to do this. Wouldn't this be the same for our thoughts and thinking?
Science is about finding evidence in support of such (reasonable) speculation.
Kant said it first! https://plato.stanford.edu/entries/kant-reason/
A professor once told me that if someone asks when a concept was first discussed in philosophy, it's always best to reply "Isn't it in Plato?"
https://plato.stanford.edu/entries/innateness-history/#PlaAr...
If that's so, then we're back to Nietzsche's Perspectivism. The experiment is limited by what the experiment entails.
This seems like it was proven ages ago with the no-free-lunch theorem.
Humans could not learn to function unless their brains encode a useful prior for learning about the world. That prior means "preconfigured with instructions for understanding the world".
The short form of the no-free-lunch theorem is that if there is no prior (i.e. all possible universes are equally likely) then for any learning problem P there are an equal number of universes that learning system A will outdo learning system B on that problem.
If not all universes are equally likely, one learning system can vastly outdo another or even most other learning systems. Not equally likely is the assumption built into brains. Without that, you can't learn effectively.
So the biology is just implementation of that general principle. The details of how that implementation works are interesting, but whether we are preconfigured for learning was never in question.
And we've proven empirically that this usable "prior" can be quite small.
What exact assumptions does human brain encode and how does it use them, however, is an area of research. We are nowhere near being able to list out all of those useful inductive biases - let alone extract them and apply them to AIs.
Seems reasonable our grey mass needs a bootloader.
IMO TFA doesn't map too cleanly onto the concept of "bootloader" (nor "microcode" for that matter). But I guess the question is, as always, can you unlock it.
Do you want to?
The range of computational processes a human brain could perform is quite large. The range of computational processes that resemble the behavior of a sane human? Far less so.
This is why Metaverse never really took off. Our brains are just ready for it.
So,
a system prompt?
I think this is obvious , otherwise how can we able to breathe once we are born ? Its same for all animals i think
There’s a difference between reacting to your environment, and being intelligent. Many top scientists working in AI, I know from firsthand experience, are disconnected from how any form of responsiveness appears in nature. It’s asinine to try to build or recreate something you don’t understand, why even bother?
Waiting for scientists to discover HUMAN.md
Makes sense, life was brute-forced.
And yet, when Chomsky says it, everyone gets very upset for some reason.
I never agreed with his views on syntaxis, but the (his?) idea that large parts of our language capabilities are innate is almost beyond doubt. Are people still arguing against it?
I think it's about the details. Chomsky argued a lot of grammar must be innate but the ability of LLMs to do grammar quite well with only a basic artificial neural network argues against that.
He should have stopped there in his career.
[dead]
[dead]
[dead]
"preconfigured" and "with instructions". i have a problem with these.
who is doing it? why the observed instructions are chosen?
One could ask the same question about any trait of any organism ... and the answer is always the same. Do you have a problem with birds being able to build nests specific to their species, or cuckoo chicks instinctively pushing the eggs of the host species out of the nest? The answer is one of the best understood facts of science, and the basis of all of biology. Why would anyone expect the human brain not to be "preconfigured" by the billions of years of environmental forces that produced it?
It's not "chosen". It is evolution. Your DNA has the metaprogram that sets up all the programs in your brain. Most of them are learning programs but you also have hardcoded programs on how to perform your bodily functions, how and when to cry, and how to suck on a tit.
just a lot of pretraining through evolution
who? The evolution. The observed instructions are also chosen by evolution
"Chosen" by an impersonal process?
You guys have entered the domain of philosophy a long time ago and didn't realize it, thinking it is still empirical science.
Anthropomorphic language about evolution is simply a convenient metaphor that eases communication ... it has no metaphysical implications.
> You guys have entered the domain of philosophy a long time ago and didn't realize it, thinking it is still empirical science.
This ad hominem sweeping generalization about people you know nothing about is so casually expressed while being so extraordinarily arrogant. Among other fallacies packed into it is a radically false dichotomy.
P.S. Oh dear ... -15 karma, numerous dead comments, and "philosophy" like https://news.ycombinator.com/item?id=30359825
Well, I won't be engaging again.
Why is this a new knowledge? And why does it take a team of whole class to find this? I think universities have ran out of research areas.
To make sure and confirm, not guess and assume
Because modern humans are cute.