Commoditizing the Newton

In this post I want to explain the vision I’m building toward and why I am building towards it.

In a nutshell, I want to commoditize the Newton by driving physical labor inputs to zero. We will do this by creating autonomous industries. Autonomous industries will be driven by Droid Capitalism — the automatic bidding on tasks. The primitives they will be built from is the droid (a smart robot). Anyone will be able to teach a Droid to do work through osmosis, infinitely scaling their tacit physical knowledge in the same way a programmer can today. In this way the entire world will participate in the current technological revolution.

The Core Problems

  • Bits > Atoms. We are over allocated to the digital economy. Even as the fundamental pieces of a whole new paradigm of “computing” in the physical world comes together the attention of the world, and the greatest ML minds, are focused on drowning the world in even more digital content. We live in a physical world and our attention should be on cheaper/better homes, medicine, transport, logistics, construction, and manufacturing.
  • Inaccessible Tech. Only a small few can take advantage and profit from abstract tools. There are vast latent spaces of untapped human potential in physical industries. What could we accomplish if a “normal” person working in the physical space could scale their idea like a programmer can in the web/mobile domains?
  • Anti-human Tech. Our tools are not built for the human body. Instead of bringing technology to people we force people into tiny boxes to interact with the digital world through a window. Causing a host of health and productivity issues.

Droids for Everyone

We will be the Microsoft of Droids by creating an end-to-end system to assimilate a robot and teach them to do work. So simple your grandma can “program” a droid and teenagers can build and deploy them.

First order benefits:

  • Zero Cost Goods: As the cost of physical labor goes to zero the cost of all physical goods will drop to the cost of it’s IP. This will let everyone live like billionaires.
  • Reduce Disparity: A programmer can write software and deploy it to infinite robots sitting in data centers. This is the key difference between the new rich and the everyone else. If we make technology accessible not only will humanity flourish but we will create unstoppable flywheel effects for our business.

Second order benefits:

  • Increase Bandwidth: We are 4D physical creatures. Our bandwidth for understanding motion is orders of magnitude greater than our ability to reason abstractly using formal math and logic. I want to tap into that and get all of mankind driving technological progress and automation.
  • Healthy Technology: Technology should promote movement and be human native rather than something we need to contort ourselves around. If we can “program” our droids with our whole body we will be healthier individuals.

Third order benefits:

  • No Reason to Fight: Because of zero-cost goods life will be so comfortable no one in their right mind will want to go to war which will drive the chance of self-destruction to zero.
  • Domesticating Space: As I explain in 🌍A Better Way to Work on Space Today the autonomous industry, and therefore autonomous robots, are the bottleneck to developing space.

The way we will do this is by:

  • creating an unambiguous, verifiable, intuitive interface your grandma could use via AR on your phone
  • self-improving ML that source the most relevant training data hands free
  • making real safety guarantees through unambiguous geometry

The Interface

Self Improving ML

Real Safety Guarantees

Business Model

There are humanoid and robotics companies emerging with billions in funding. Where do we fit in? These companies are just the tip of the spear. There will be 100,000 more companies just like them solving for their own niche in the industrial ecosystem.

We have no desire to compete by building our own robot. These companies are great indicators that our product is desirable.

We will use a prosumer business model that enables people to build and train their own droids rather than buying a 6-figure droid off the shelf from someone else. We will be the roblox of droids.

Unfair Advantages

  1. A confluence of rising tides that together create a tsunami.
  2. 100x better product. In order to build a safe droid of any kind it is almost impossible without millions in funding and top talent in robotics and ML.
  3. Monopoly. The most accessible droid will create an unbreakable monopoly due to flywheel effects. Because we facilitate, but do not make hardware, we will scale 10-100x faster than any point solution robotics out there while retaining the flywheel advantages.
  4. Founder — I’ve been putting this jigsaw together since 2017 with a set of experiences that less than 1/100 founders in this space have.

Market Rising Tides

There are many hi-profile robotics companies building droids, from Optimus, to Figure1 and perhaps drone delivery and it is no surprise. There are two drivers in the US:

  1. Manufacturing Re-shoring: Covid showed us our supply chain weakness, coupled with our difficulties with China means there is incentive to actually become decent at physical production again.
  2. Physical labor shortages: we just don’t work like we used to and there aren’t enough of us especially with the silver wave.
  3. At the same time due to innovation in other markets, like genAI, fundamental research is exploding. These techniques can be adapted to our domain.
    1. Accessible AR: AR has gotten good — really good. We can build an interface for programming robots that your grandma could use.
    2. Accessible Robot Hardware: Hardware is dropping rapidly in cost and we can get useful robots for the cost of an Apple Pro or other consumer grade device. From chips like Nvidia Jetson to chassis fabrication.
    3. Accessible Edge Inference: Think of Apple’s on device edge compute push for inference.
    4. ML Techniques: This underpins everything from AR interfaces, power object recognition, segmentation, representation, auto-curriculum generation.
    5. Simulation: we have seen the rise of platforms like Nvidia, and their omniverse toolchain. Not to mention the Mujoco purchase by DeepMind. Zero-shot sim2real is now possible.
    6. Open-Source Software: Ecosystems like ROS 2.

Tech has gotten good enough that more money and players are getting involved so tranferable tech will only get better across the board which is better for everyone.

Product

How We Will Beat Billion Dollar Companies

What are our product advantages? Think of the alchemy of Mobile Aloha, , and an AR enabled “mouse and keyboard” interface for robots that your grandma could use.

  • N hardware vendors. We do not focus on building droids, though we will provide initial dev kits as a proof-of-product. Instead we will be infrastructure. Our customers will make droids, our platform will enable them to be mobile, and the end consumer will program the applications. We bookend the process by providing interface tools
  • Instant global access. You can use your phone to program your droid. Leveraging the existing hardware and platforms using AR enabled devices in the Apple/Android ecosystems
  • Droid capitalism — since our programs are droid agnostic we can create a capitalistic system of droids. We believe droids will go the way of the utility.
  • Self-improvement — due to how we have designed our interface it is possible for the droid to focus only on the training that will help the most

Monopoly

Flywheel effects of the prosumer model

Insights

  • Chaos & complexity theory

What do I see differently? What’s my advantage that will enable me to build this future and not someone else? I hold a number of contrairian beliefs that have led me to predict the failures of autonomous systems since 2019 — especially self-driving cars and mobile delivery. I do believe the next five years will see the first robust systems get deployed due to a much needed switch in approach and model architecture.

Here are the core insights and secrets that are as relevant now as they were in 2019 when I first examined and abandoned the field:

  • The Scaling Thesis is lazy. We are so obsessed with what has worked we are blinded to the real opportunity. With $1 billion training runs on the horizon it’s clear we are addicted to big data as a simple way to get performance and this causes us to overlook the next paradigm shift.
  • There is no general purpose droid body. A body is just a tool. There is no one tool for all situations. The general purpose robot is a myth because the point of generalization is in the “mind” of the machine not in this or that morphology.
  • The next paradigm shift is small models. Why? Because what matters is the smallest possible example of self-learning machines that can efficiently seek out its own training data. Everything else is irrelevant. Chaos theory and complexity theory tells us that complex systems require multiple core drivers to create a bifurcation in behavior that oscillates around a forward vector of the derivative of learning. Transformer architecture is one component, high signal feedback is another, non-synthetic self-data creation is another — if we put the core pieces together just right we will create alchemy and a non-linear explosion of self-improvement.
  • Ignore language and focus on geometry (to start). The bottleneck to learning cheaply and effectively is tight, relevant feedback. And yet we try to brute force learning with garbage in and legions of humans on the backend doing RLHF. Instead, if we focus on geometry we can provide:
    • mathematical ways for machines to teach themselves through synthetic data
    • trivial interfaces for humans to peek into their “minds” (and understand at a glance what is happening)
    • a trivial to understand interface
    • a way to provide safety guarantees.
  • Generalization is easier than specificity for 4D tasks: In 2019 I knew and claimed that self-driving cars would fail because of the keyhole problem. When operating in the 4D environment object transforms are tightly coupled — the same skills transfer. Learning a wide variety is more efficient than training.
  • Embrace multi-body over single body. The super power of the silicon stack over the carbon stack is that it can treat bodies like tools. There is no coupling. Most robot companies, even those with billions in funding, are making the mistake of focusing on one hardware form factor — think humanoid robots or the robot arm company. The fallacy is that every single creature on the planet runs the same basic 4D navigation wetware from squirrels to humans. The specific form is the least important aspect of general physical intelligence.
  • Crowdsourcing: Instead of building one droid, give the tools to let the entire world build their own droids and create a monopolistic flywheel of self-improvement with 7 billion employees building on the platform.
  • Tight Focus: Since we are focusing on the OS we are spread across fewer domains enabling us to build better product faster.
  • Focus on verifiable interfaces: Language does not produce verifable actions that can be watchdogged. The way we improve the world is by making exponential technology accessible. We need to make “programming” human native by making it expressed in visual geometry (shapes).
  • Safety Guarantees Matter: most companies are focused on end-to-end neural networks whose plans and actions cannot be verified. Because we use non-ambiguous intuitive geometry for our interface we can make guarantees about droids verifying their own performance
  • We can not trust ML machines.
    • Making machines be “human like” is perverse and dangerous. The silicon stack for intelligence is entirely divorced from our own.
    • Label the alien as an alien will build trust. We believe labelling this in our branding and identity will set us apart and breed trust. This is not only the safe thing, but its the honest thing, and this will create consumer trust.

Founder

cluster of insights that have been architected over the last decade has come together to create alchemy. Rooted in biology, neuroscience, evolution, chaos theory, manufacturing, robotics, software, and product design. Validated by working at first class robotics acquired for $400 million, R&D at Fortune 100 manufacturer, and the leading XR company — plus founding my own startup. I do not see a better founder option only tradeoffs. The only way I can see to improve is if I had more capital, had worked directly in ML, or had lucked in to a halo effect from a high profile acquisition or project.

Outlook & Asks

I’m currently heads down to put together the jigsaw pieces and produce further indications. You can follow along with my DevLog on my blog.

todo link to devlog on blog, or spoke?

Roadmap

  • Alpha — Demo & Proof of Concept
    • Target completion: Q3 2024-Q1 2025
    • Funding stage: Pre-Seed < $5 million
    • Deliverables
      • Full Simplified System in Simulation
      • Z0 - (stretch) “mini” hardware dev kit
  • Beta — Working Prototype & Early Usage
    • Funding stage: Seed < $20 million
    • Target Completion: Q3-Q4 2025
    • Deliverables
      • Core team
      • AR Interface v1
      • Pick and Place Hardware dev kit v1
        • Z1 - stationary
        • Z2 - mobile
  • MVP — Product
    • Funding stage: Series A
    • Target Completion: ???
    • Deliverables
      • Foundry — bring your own hardware

Asks

I am looking for the following things things:

  • DIY roboticists interested in making droids at home
  • Founders with 8+ figure exits or companies, especially deep tech domains
  • technical experts for advisory board
    • sim2real transfer
    • AR interfaces
    • synthetic representation and modeling of 4D environments
    • decision transformer and auto-curriculum generation
  • team members (near term)
    • software engineer specializing in AR kit or related tooling
    • ML engineer with working knowledge of sim2real transformer and decision transformers
  • I may bootstrap until Seed or Series A but there is a chance I will raise a pre-seed in 2024. I haven’t decided yet. It will depend on whether I think it will benefit recruitment efforts and/or the estimates of compute costs.

Appendix

Human Native Compute

The last decades has seen a great migration to the invisible. Physical objects have been evaporating replaced by screens on a tablet. This paradigm is going to reverse as we see an explosion of the digital breaching into our physical spaces. This will take the form of robots, light/laser/hologram projections, a long tail of specialized devices/appliances, AR overlays, and physical intelligence embedded into everything. This is a good thing because

Not many know that in 2019 I created a corporation called Surrogates.ai but it was far, far too early. I knew then that we ML was addicted to big data and that narrow AI like self-driving would never work in a vacuum with SOTA techniques until something fundamentally changed. So, I sat, and I watched, and I waited for the key factors to go from red to yellow to green.

You can imagine having a big list of these factors and assigning a red, green, yellow value to each of them. Now, they are all green or going to green over the next 2-3 years. The world will lag behind, dragged down by the inertia of big data techniques and the quick success it offers. It may work out if they can get enough progress and capital to reinvent themselves with tiny auto-didactic models. However, most of the world will be stuck in the software/web world and suffer from inertia. By design I am unencumbered by any legacy system and so I will be able to synthesize just the right puzzle pieces.

After looking at ~1,000 papers in the last 5 months I am certain all the technical hurdles to building that tiny auto-didactic model that can power any robot body is possible. Further, it is enabled by a confluence of dozens of key technologies that have just matured or are maturing rapidly. After paying close attention to every hyped technology of the decade I have never believed in anything more.

Let’s talk about concrete plans for this year. There are many details and questions to figure out. Not only are the technical hurdles fun, but there are the challenges of finding product market fit and handling the transition. Not easy in any environment. Ultimately, with deep tech and a new market my job is to sell the vision and demonstrate potential — ideally with product of some kind. Then demonstrate traction, of any kind. I do that by presenting indications.

Here is the rough progression for near term customer discovery and validation:

  • Proof of concept demo of the AR interface
  • Traffic tests against landing page
  • Pre-orders / Kickstarter test
  • Dev Kit (lo-fi MVP)

In most cases, I would start with the traffic test, but in this case a demonstration will be worth 10,000 words. I can build a splash experience to serve as a trailer with which to drive activations and inform my design and explore AR capabilities at the same time.

♾️Droids Master Plan