One of the things I have always found amusing about artificial intelligence is that we took inspiration from biology to create neural networks, and then implemented those networks on computing architectures that have almost nothing in common with the biological systems that inspired them. The brain does not have a GPU (despite GPU have been really useful so far!), does not continuously shuttle data between separate memory and processing units in the way conventional computing systems do, and does not stop every evening to retrain itself on everything that happened during the day. It learns while operating, processes enormous quantities of sensory information continuously, adapts to a changing environment, and somehow accomplishes all of this while consuming about as much power as a dim light bulb, which remains, after all these years, a somewhat embarrassing comparison for computer engineers.
This does not mean that we should literally reproduce the brain in silicon, because biology has had a few billion years to optimize its solution and has also accumulated a number of design choices that I am not sure any engineer would willingly put into a product specification. What biology does demonstrate, however, is that intelligence and the physical system implementing that intelligence do not have to be separated as cleanly as we have become accustomed to doing in computer science. In biological systems, memory, computation, learning, timing and connectivity are deeply intertwined, and this relationship has influenced much of my thinking throughout my career, first as a computational neuroscientist and later as an entrepreneur trying to build AI that had to work inside robots, cameras, drones, industrial equipment and other physical systems where power, latency, cost and memory eventually become very concrete problems.
Biology never really separated hardware and software
One of the great achievements of modern computing is abstraction. A software developer can write a program without thinking about transistor geometry, memory cells or the electrical details of the processor, because layer after layer of architecture hides those complexities and creates the illusion that computation is a relatively uniform resource. This abstraction has been extraordinarily productive, but it also encourages us to think of the algorithm as something independent from the machine running it.
Biology took a different path. A synapse stores information and participates in computation; the connectivity of a neural system influences what that system can represent; learning physically modifies the structure through which future computation will occur; and timing, locality and communication are part of the computational process rather than implementation details hidden underneath it. This intimate relationship between architecture and computation was one of the motivations behind neuromorphic computing and behind some of the work we carried out at the Boston University Neuromorphics Lab, including research connected with DARPA’s SyNAPSE program.
At the time, neuromorphic computing was still a specialized field populated by neuroscientists, engineers and chip architects who were trying to understand whether biological principles could inspire more efficient artificial systems. Today, many of those same questions are reappearing in much more commercial language because AI is increasingly moving away from centralized computing infrastructure and into machines that have to operate continuously in the physical world.
AI becomes a different problem when it gets a body
An AI service running in the cloud enjoys conditions that are, from an engineering perspective, rather luxurious. Power is available, cooling is available, memory and storage can be expanded, and additional computation can often be added when necessary. Once the same intelligence is placed inside a drone, a robot, a vehicle or an industrial machine, the situation changes because the machine cannot simply ignore physics.
A drone has a battery, and therefore every additional watt has a direct relationship with flight time. A robot processes continuous streams of sensory data and may need to react within milliseconds, which makes latency part of its behavior rather than a line item in a benchmark. An industrial camera may have a processor that was selected years before the AI application existed. A factory may not allow production images to leave the premises, either for privacy, security or practical reasons. A machine may have to continue operating even when connectivity disappears completely.
During the development of Neurala, we encountered these kinds of constraints repeatedly because our technology was deployed across tens of millions of devices and had to run on very different computing platforms. We could not assume that every customer had the same processor, the same power budget or the same willingness to redesign an entire system in order to accommodate our software. The real world has a remarkable ability to convert an elegant neural architecture into a practical discussion about memory bandwidth, thermal limits and bill of materials.
For this reason, I have always thought that Edge AI is less about geography than it is about constraints. The fact that computation happens locally is important, but the more interesting question is why it must happen locally and what those constraints imply for the architecture of the system.
Hardware/software co-design eventually becomes a business question
Once power, memory, bandwidth, latency and privacy become important, the distinction between algorithm design and hardware design becomes less clean. A model that performs perfectly well on one processor may become impractical on another because of memory traffic or numerical precision. A small change in quantization may alter the accuracy of the network, but it may also change the amount of memory required, the energy consumed and the types of devices on which the model can run. A decision that looks like an optimization inside a research environment can therefore change the commercial viability of the final product.
Over the years at Neurala, we worked with companies across the semiconductor ecosystem, including Qualcomm, Sony Semiconductor Solutions, Movidius, Lattice Semiconductor and others, and each of these collaborations reinforced the same lesson in a slightly different way. The algorithm could not be treated as an isolated object because its behavior depended on the computing substrate, while the usefulness of the hardware depended increasingly on the workloads it could support efficiently.
This interaction became particularly interesting when we moved into embedded and industrial AI, because hardware efficiency began to influence customer economics directly. If an AI capability could run on a processor already present in a customer’s product, the integration process might remain relatively simple and the cost structure might be attractive. If the same capability required a different board, more memory, additional cooling and months of engineering effort, the technical improvement had to be very significant before anyone would consider deploying it.
This is one of the reasons I have become increasingly interested in hardware/software co-design. It is easy to describe co-design as an engineering discipline, but in many applications it becomes a product and business discipline as well, because the architecture of the computation influences which markets are accessible, what the final system costs and how easily the customer can adopt the technology.
Efficiency always finds its way into economics
Engineers naturally discuss efficiency using engineering metrics such as power consumption, memory bandwidth, throughput, latency and utilization. These are useful measures, but after running a company for many years I learned that almost every one of them eventually appears somewhere in the economics of the product.
Power consumption influences battery size, thermal design and operating cost. Memory influences processor choice and bill of materials. Latency determines whether a system can perform certain functions at all. The need to move data to the cloud creates bandwidth and infrastructure costs and, in some environments, security and privacy concerns. A model that performs slightly better but requires significantly more expensive hardware may therefore be technically superior while producing a worse product.
This is particularly important because much of the AI industry has become accustomed to evaluating progress using benchmarks that isolate the model from the rest of the system. Accuracy, parameter count and throughput are all important, but they are incomplete descriptions of an AI product once that product enters the physical world.
The customer ultimately buys a system that has to function within a power envelope, a cost structure, a latency requirement and a deployment environment. At some point, FLOPS become dollars, memory becomes dollars and watts become dollars, which is where architecture stops being only a technical consideration and begins to influence corporate strategy.
Semiconductor partnerships are more strategic than they first appear
When Neurala first began working closely with semiconductor companies, my instinct was to think of those relationships primarily in technical terms. The question was how to make our AI run efficiently on a particular device, how to adapt models to the characteristics of the architecture, and how to reduce the engineering effort required to move from prototype to deployment.
Over time, I realized that these partnerships could affect much more than performance. A semiconductor platform is often designed into many products across multiple markets, which means that an AI capability that maps naturally onto that platform can inherit part of its distribution and ecosystem. If the software works well on hardware that customers already understand and trust, the path to adoption can become much shorter.
This is especially important in industrial and embedded markets, where systems tend to remain in production much longer than consumer software cycles would suggest. Customers do not replace equipment every eighteen months simply because a new AI architecture has become fashionable, and they are understandably reluctant to redesign a working platform in order to accommodate a software supplier.
As a result, the relationship between AI companies and semiconductor companies can become strategic in both directions. The AI company gains access to a broader deployment base and better economics, while the semiconductor company can offer its customers higher-value capabilities that make the hardware more attractive. The most productive relationships occur when both sides are willing to modify their assumptions rather than expecting the other layer of the stack to absorb all of the complexity.
Scaling has been extraordinary, but it is not the only design direction
The success of large neural networks has demonstrated something important about intelligence. When models, datasets and computational resources become sufficiently large, systems begin to exhibit capabilities that were extremely difficult to predict from smaller versions of the same architecture. This has been one of the most impressive developments in AI, and I do not think it should be minimized simply because alternative approaches may also exist.
At the same time, the success of scaling does not imply that every future intelligent system should become a larger version of the same idea. Engineering always operates within physical constraints, and the moment AI moves outside large centralized infrastructure, the cost of computation becomes much more visible.
Biology provides an interesting counterexample because natural intelligence evolved under very severe energy constraints. An animal that required a data center to recognize food would probably have experienced a rather short evolutionary history. Energy efficiency was therefore never an afterthought for biological intelligence; it was part of the problem from the beginning.
This is one reason I believe ideas such as sparse computation, event-driven processing, continual learning, local memory and specialized accelerators are becoming increasingly relevant again. Many of these concepts have existed for decades inside neuromorphic computing and related research communities. What has changed is that the commercial environment now contains many more applications where efficiency is no longer optional.
Twenty years ago, some of these ideas were interesting because they resembled aspects of biological computation. Today they are interesting because a robot, a camera or an industrial machine may simply require them.
Intelligence will probably become heterogeneous
I also doubt that the future will be organized around a simple choice between cloud AI and Edge AI. It seems much more likely that intelligence will become distributed across different layers of computation, with different architectures performing the tasks for which they are best suited.
Large centralized models may provide broad reasoning, knowledge and capabilities that justify significant computational expense, while local models inside machines handle perception, rapid decisions and functions where latency or privacy make remote computation impractical. Specialized hardware may execute narrow operations very efficiently, and sensors may increasingly perform part of the computation before the data even reaches the main processor.
Robotics makes this architecture particularly easy to imagine. A robot might use a large model for planning, interaction or higher-level reasoning, while relying on local systems for motor control, collision avoidance, vision and other tasks that cannot tolerate the delay or uncertainty of a network round trip.
Biological systems once again offer a useful analogy, although I would be careful not to push the comparison too far. Intelligence in biological organisms is organized across different structures, levels and timescales, and not every signal is processed in the same place or by the same mechanism. Artificial systems may eventually adopt a similar heterogeneity because it is efficient engineering, whether or not anyone chooses to call the resulting architecture brain-inspired.
Physical AI brings architecture back into the center of the discussion
For many years, AI research benefited enormously from general-purpose accelerators becoming faster and more capable, which allowed researchers to concentrate on algorithms while assuming that computing infrastructure would continue to improve underneath them. As AI becomes increasingly embedded in the physical world, that assumption becomes harder to maintain because physical systems impose constraints that cannot always be solved by adding more compute.
Robots, industrial machines, autonomous devices and intelligent sensors may require very low latency, very low power, persistent operation, local adaptation or strict privacy. These requirements force us to ask architectural questions that are easy to ignore in a data center: where should memory live, how much computation should happen close to the sensor, what should be implemented in software, what benefits from specialized hardware, and whether the neural architecture itself should change because of the characteristics of the machine that has to execute it.
These are questions I was asking as a computational neuroscientist before AI became the enormous commercial industry it is today, and I find it interesting that the expansion of AI into physical systems is bringing many of them back. The terminology has changed, which is normal. Technology has an impressive ability to rediscover old ideas with new names, especially once there is a market attached to them.
The underlying problem remains familiar: intelligence cannot be designed indefinitely without considering the physical system in which it must exist.
The boundary may be where the interesting things happen
After spending many years working across neural algorithms, neuromorphic computing, embedded AI and semiconductor platforms, I increasingly believe that some of the most interesting developments in AI will happen precisely at the boundary between the model and the machine.
Traditionally, that boundary has been treated as an interface between separate disciplines. The AI researcher creates the model, the compiler team makes it run, the semiconductor company builds the processor, and the product company integrates everything into a final system. This division of labor works extremely well when the architecture is mature and when computing resources are plentiful.
It becomes less effective when the goal is to create new forms of intelligence under tight constraints, because the assumptions made at one layer directly affect what is possible at another. The people designing the algorithm benefit from understanding something about the hardware, while the people designing the hardware benefit from understanding the structure of the workload.
Neuromorphic computing explored this relationship long before it became commercially fashionable, and I suspect that we will see much more of this thinking as AI moves into the physical world.
I do not expect every future architecture to resemble the brain, and I would be suspicious of any argument suggesting that copying biology automatically produces better intelligence. The more useful lesson from biology is simply that highly capable computation can exist in forms very different from the architectures that dominate computing today.
For an engineer, that is already an important result, because it tells us that the design space is much larger than the small part we have explored so far.
Somewhere between the neuron, the algorithm and the transistor, there are still many architectures waiting to be discovered, and after spending much of my career around that intersection, I continue to think that this is where some of the more interesting surprises in AI are likely to appear.