Modular has raised $100M to fix AI infrastructure

Chris Lattner and I started Modular to help improve AI infrastructure for the world, and to enable the next wave of AI innovation to be truly unlocked on the world's hardware.

We are so excited to announce this $100M raise, and beyond proud of what our world-class team, incredible customers and partners have enabled us to achieve. Our AI Engine is the world's fastest and has a strong list of customers lining up for its unparalleled performance and usability, and our new programming language in - Mojo 🔥 - already has a developer community of >120K+ developers in just 4 months.

This round was led by General Catalyst, in Deep Nishar&Christopher Kauffman, and filled out by existing investors in GV (Google Ventures) with Dave Munichiello, SVA in Steven Lee&Ronny Conway, Greylock in Saam Motamedi and Factory - amazing leaders and incredible people.

We are so fortunate to work with them, and their belief, to build and create change in the world.And of course, changing the world is never easy - but we are so incredibly determined to continue to do so. AI is so important to the future of humanity, and we feel a great purpose to truly improve AI's usability, scalability, portability and accessibility for the worlds developers and enterprises.

Join us on this incredible journey, and let's change the world together 🚀! You can read more on the Modular blog.

Founding Modular & Raising $30M

We are bringing together the world's best AI infrastructure talent to improve AI production development and deployment.

After working for years in the AI/ML space - I’ve left Google and decided it's time for a new approach to building machine learning infrastructure. Chris Lattner and I, along with an incredible team of talented architects, engineers and product leaders are teaming up to rebuild it from the ground up and truly help the world of AI.

We building a next generation AI developer platform, and we are proud to partner with @GVteam, @GreylockVC, Factory, SV Angel and notable angels who are funding our $30M first round of funding. We spoke with Dave Munichiello from GV about the opportunity here. The next generation of product breakthroughs will be powered by production quality infrastructure that brings together the best of compilers and runtimes, is designed for heterogeneous compute, edge to datacenter distribution, and is focused on usability. Unifying software and hardware with a "just works" approach that will save developers enormous time and increase their velocity.

Having worked for many years in the AI space at Google, we have and are continuing to assemble the world's best AI infrastructure team. You can read some of the challenges the industry faces, in our opinion, via our blog post here: The Case for a Next-Generation AI Developer Platform. We are hiring for numerous roles - please apply via Modular Careers.

We're excited to showcase what we have been building and designing later in the year. You can checkout a video we put together below.

We are incredibly excited about the mission before us and if you are interested in joining us to change the world - just reach out via www.modular.com. We’re hiring everywhere. The future is super exciting and bright!

Help Jason find his Rhythm

Learn how TensorFlow Lite Micro helped Jason find his rhythm after loosing his arm - this is story and showcasing the power of AI.

Many years ago, Pete Warden, Raziel, Rocky, Sarah, Andy and I, with a small and incredible team at Google, founded TensorFlow Lite Micro for the world as we recognized the importance of executing machine learning on microcontrollers. We have continued to scale and contribute significant improvements to TF Lite for Microcontrollers over the years unlocking a multitude of use cases from speech, to person detection, to a multitude of audio detection use cases and more. We have seen the creation of cascading networks where the front of the pipeline is a very low power microcontroller running a tiny inference model all the way through to multi-model pipelines firing up a more significant application processor.

Here is just one incredible story where the team from Georgia Institute of Technology under Gil Weinberg, took the the TensorFlow Lite and TensorFlow Lite Micro work and drove an inspiring use case to help Jason find his rhythm. Watch the video below to understand how TensorFlow Lite is being used to empower human augmentation and empower people to rediscover that anything is possible. If you're running short on time - here's how it works.

As Stephen Hawking once said:

“Remember to look up at the stars and not down at your feet. Try to make sense of what you see and wonder about what makes the Universe exist. Be curious. And however difficult life may seem, there is always something you can do and succeed at.

Saving the Great Barrier Reef

Helping save the Great Barrier Reef using TensorFlow and the power of Machine Learning

Coral reefs are some of the most diverse and important ecosystems in the world - both for marine life and society more broadly. Not only are healthy reefs critical to fisheries and food security, they provide countless additional benefits: protecting coastlines from storm surge, supporting tourism-based economies and sustainable livelihoods, and pushing forward drug discovery research.

2 years ago, at a lunch and meetup with Martin Wicke and Brano Kusy - we decide to work together and team up to see if Google could contribute to saving the great barrier reef from Crown of Thorns starfish and identifying other sea life. As an Australian - I knew of the importance of this mission for the world and for future generations to experience the wonder of the reef, and I decided to lean in heavily and help champion this effort at Google, helping to drive it forward. What followed - was a multi-step journey, with an incredible team and group of folks at Google and CSIRO, helping Brano's team end-to-end label, train, scale and deploy, to their boat mounted hardware, a new solution for rapidly identifying COTS (and many other things) in the Great Barrier Reef. You can see our research paper for the data here.

We launched a Kaggle competition, to help crowd source the final approach and enable this to be executed live on the Great Barrier Reef, and we're open sourcing it for the world as well. You can checkout my narration in the video below and see the original TensorFlow post.

Reefs around the world face a number of rising threats, most notably climate change, pollution, and overfishing. In the past 30 years alone, there have been dramatic losses in coral cover and habitat in the Great Barrier Reef (GBR), with other reefs experiencing similar declines. In Australia, outbreaks of the coral-eating COTS have been shown to cause major coral loss. These outbreaks can strip a reef of 90% of its coral tissue. While COTS naturally exist in the Indo-Pacific ocean, overfishing and excess run-off nutrients have led to massive outbreaks that are devastating already vulnerable coral communities.

Controlling COTS populations is critical to reducing coral mortality from outbreaks. Google has teamed up with CSIRO to supercharge efforts in monitoring COTS using artificial intelligence. This is just the beginning of a much deeper collaboration and we, along with the Great Barrier Reef Foundation, are extremely excited to invite you, our global ML community, to help protect the world's reefs.

MLIR - Open Source Infrastructure for the world

MLIR offers new infrastructure and a design philosophy that enables machine learning models to be consistently represented and executed on any type of hardware.

MLIR is a new compiler stack that I have the privilege of being the Product Manager for at Google. It fundamentally takes a completely different perspective on existing technologies available in the marketplace by producing a multi-level intermediate representation that combines high level optimizations with lower level code generation in a way that hasn't been done before. I have had the privilege to work with Chris Lattner and a talented team of many other folks in building out this technology and look forward to the enormous impact it is going to have on machine learning in the years ahead. Given that we, Google, are an AI first company - even Sundar was happy about the news. Below is a repost of the original announcement that we posted to the main Google blog.

Machine learning now runs on everything from cloud infrastructure containing GPUs and TPUs, to mobile phones, to even the smallest hardware like microcontrollers that power smart devices. The combination of advancements in hardware and open-source software frameworks like TensorFlow is making all of the incredible AI applications we’re seeing today possible--whether it’s predicting extreme weather, helping people with speech impairments communicate better, or assisting farmers to detect plant diseases.

But with all this progress happening so quickly, the industry is struggling to keep up with making different machine learning software frameworks work with a diverse and growing set of hardware. The machine learning ecosystem is dependent on many different technologies with varying levels of complexity that often don't work well together. The burden of managing this complexity falls on researchers, enterprises and developers. By slowing the pace at which new machine learning-driven products can go from research to reality, this complexity ultimately affects our ability to solve challenging, real-world problems.

Earlier this year we announced MLIR, open source machine learning compiler infrastructure that addresses the complexity caused by growing software and hardware fragmentation and makes it easier to build AI applications. It offers new infrastructure and a design philosophy that enables machine learning models to be consistently represented and executed on any type of hardware. And today we’re announcing that we’re contributing MLIR to the nonprofit LLVM Foundation. This will enable even faster adoption of MLIR by the industry as a whole.

MLIR aims to be the new standard in ML infrastructure and comes with strong support from global hardware and software partners including AMD, ARM, Cerebras, Graphcore, Habana, IBM, Intel, Mediatek, NVIDIA, Qualcomm Technologies, Inc, SambaNova Systems, Samsung, Xiaomi, Xilinx—making up more than 95 percent of the world’s data-center accelerator hardware, more than 4 billion mobile phones and countless IoT devices. At Google, MLIR is being incorporated and used across all our server and mobile hardware efforts.

Machine learning has come a long way, but it's still incredibly early. With MLIR, AI will advance faster by empowering researchers to train and deploy models at larger scale, with more consistency, velocity and simplicity on different hardware. These innovations can then quickly make their way into products that you use every day and run smoothly on all the devices you have—ultimately leading to AI being more helpful and more useful to everyone on the planet.

The career opportunity matrix

The biggest mistake I constantly see new founders make is misjudging the matrix that almost everyone goes through when making a career (and most often life) changing decision. I certainly didn’t appreciate this as much as I should have when I was a founder and I urge anyone reading this post to deeply consider aspects highlighted in this post.

Both at my previous startup and with the many subsequent startup founders that I’ve talked too since joining Google — the biggest mistake I constantly see new founders make is misjudging the matrix that almost everyone goes through when making a career (and most often life) changing decision. I certainly didn’t appreciate this as much as I should have when I was a founder and I urge anyone reading this post to deeply consider aspects highlighted in this post.

When you ask someone to join your company —you will almost immediately misjudge the impact it will have on their life. You are asking someone to ultimately change the value they attribute to their time in what they currently do — in order to encapsulate themselves in whatever it is you do. The value of this utility is ultimately the most valuable thing any person has — the opportunity to spend their time when and how they see fit.

If you are unable to relate to the person, inspire the person, compensate the person or truly enable them to believe in the growth opportunity you present — why would they offer up their most precious asset to work with, and ultimately, for you? Talented people, truly great people — understand the value of time. Its a common interview question of mine to press people on explaining to me how their organize and attribute value to it. The reality of the world is — most people don’t attribute enough value to their time and waste it frequently.

So as a founder, understand the importance of time. Its almost unequivocal at this point that building out an all star team is going to be the single most important aspect attributable to your companies success — a great team is always going to be able to derive a solution in good times and bad. But consider this — when you ask, or pitch, someone to join you — do you really appreciate the matrix of decisions that they are going through to join your company. Why would anyone want to spend their time with you?

In my experience to date, from startups through to Google, here are the only four major aspects that matter to people who attribute real value to their time. This is their opportunity cost matrix:

1. Family — In a personal capacity, most people will not sacrifice their family for anything. It is always what is prioritized in people’s lives above all else — it is uncompromising. Equally, in a professional capacity, people want to feel that they have a real connection to their work mates and their team. Usually, a few extroverted people bind a team together and help it thrive. You know that you have a team that feels this way when someone leaves — you actually miss that person when they do. If you miss the importance of family in someones life, or you overlook it, you do so at extreme peril more often than not.

2. Remuneration — Typically, no one works for free. People value their time and they deserve to be compensated for their skill. Cash is almost viewed to be infinitely more valuable than any equity you will provide someone. Great people need to be compensated well, or at least very fairly, from a cash standpoint and even better from an equity one. You might treat your equity at a premium, but asking someone to leave a top 10 tech company to join your team is asking them to move from a world of liquidity to a long life without it. Just look at the giants like Airbnb and Uber — more than 10 years later many employees still remain without any liquidity. So remember, more often than not I have found that liquid cash is king and that people work to care for their families.

3. Growth — Everyone wants to advance their careers in some capacity — whether it be personal knowledge, leadership or the chance to be promoted. Not everyone cares about the later — some are very satisfied with a growth in their own knowledge without significant change in their ladder ranking. Equally, some do. Working on a product and with a team that facilities and enables growth ultimately leads to higher retention — promote people quickly who do a fantastic job and its often easy for you to tell who your organization could not live without. Do not fall into the trap of rewarding them “down the line” — the simple reason being that great people are quickly acquired and this matrix is more than likely being pitched to them at numerous other companies as well.

4. Purpose — A purpose or mission is typically much bigger than what you are working on today. It’s the vision that you should be proud to tell your friends and family and is much larger than the current iteration of your product or company. Teams that don’t have a mission don’t know where they are going and aren’t connected through the shared bond of a greater goal or purpose. A mission is why you get up and come to work everyday and is typically something you really believe in. It isn’t captured by in metrics but in how you are trying to put a dint in the world. If you cannot inspire someone to believe in the purpose of what you are doing — then regardless of anything else — they will never truly ride the rollercoaster with you.

Everyone deserves to feel that they can grow their own careers and have a sense of personal achievement as they define it. Equally, they should be able to tell the world what they are passionate about, why they are working on it and have a defined sense of purpose. Everyone has to point north and believe in what they are doing — in fact

If everyone on your team can’t tell a friend very simply what problem your product is solving, why it’s unique, and why you will win, you might as well stop and go home now.

As an anecdote, and in this light, when I previously pitched to Alibaba’s investment arm and asked their investment partners why they worked at Alibaba. The response was the most impressive one ever heard:

“At Alibaba, we aim to build the future infrastructure of commerce. We envision that our customers will meet, work and live at Alibaba, and that we will be a company that spans over three centuries. We take a very long approach in every investment we make and everything we do.”

Your ambitions and goals should be as easily declared when anyone on your team is asked what it is you do — a clear vision, a clear sense of purpose, a passionate sense of worth and a guiding north star. Without it — why should anyone join you?

“And most important, have the courage to follow your heart and intuition. They somehow already know what you truly want to become. Everything else is secondary.” — Steve Jobs, 2005

Presenting at Google I/O

Dancing on stage at Google I/O 19 in front of 15,000 was a fun experience and inspiring. The video is below and I start at 42:55 into it.

It was super fun to speak at Google I/O Developer Keynote this year about the amazing work the TensorFlow Lite team has done on scaling on-device ML globally. You can checkout the video of the presentation below which showcased a new interactive form of video and the ability to scale that to a range of new device platforms.

The TensorFlow Lite team, combined with other internal Google Research and performance teams, showcased whats now possible utilizing on-device ML with a Pixel 3 and the GPU delegation improvements that we have previously announced. Here’s the video giving a deeper dive into the making of Dance Like.

Supportive Confrontation

Supportive confrontation is a methodology adopted by respectful and highly constructive individuals and teams to ultimately push the boundaries of self-improvement. David Bradford (Stanford) and Allen Cohen ultimately coined the termed in their book Power Up and David utilizes this in his class on High Performance Leadership.

Supportive confrontation is a methodology adopted by respectful and highly constructive individuals and teams to ultimately push the boundaries of self-improvement. David Bradford (Stanford) and Allen Cohen ultimately coined the termed in their book Power Up and David utilizes this in his class on High Performance Leadership.

In my career, and notably at Google, we utilize supportive confrontation to improve as individuals and strengthen our collective team in working to continue to build world class products.

So what is it?

The very definition of “confrontation” presumes some measure of hostility or disagreement between parties. In its purse sense, it is appreciated why the name alone instills some level of unsurety in many individuals who undertake this method of feedback adoption. However, the fundamental premise of “supportive confrontation” is a balanced combination of the former with the later - to deal with personal growth in a direct and honest way. Indeed, in the absence of such direct and honest feedback, a host of eventualities ultimately occur including emotional instability, poor accountability and ownership, individual conflict and poor team performance.

The focus for supportive confrontation is to initiate conversations with individuals that you have an obvious mutual respect for and whom would be open to receiving such direct and honest feedback. You should recognize that not all work colleagues (or friends) can undertake this feedback methodology without immediately reacting in a defensive manner — so choosing the nature of your audience is a critical one. The ultimate goal is, through mutual respect of person or people providing such feedback, to abject any form of personal emotion and consider the feedback for what it is — open and honest. In order to improve as an individual, one has to accept all forms of criticism in life and ultimately listen intently, reflect and channel this feedback forward.

Here are some broad level pointers:

1. Open transparency from the beginning — If you have asked an individual, or a group, to engage in supportive confrontation, you should illustrate that the purpose of such feedback is to improve and not to engage in personal attacks. The art of this form of feedback is mutual respect for each person and explaining this from the outset is critical to it being successful. Explain from your own perspective how and why this feedback system has been useful in the past and ensure that everyone can empathize with this point of view.

2. 3 Top Aspects, 3 Bottom Aspects — Provide each person with three things they are doing well, and three things that they ultimately need to improve on. Feedback should always be direct and actionable — including real examples of what you have noticed and when further helps an individual to consider how they can improve. Good and bad feedback without observable examples makes it difficult for people to reflect and consider — so quantifying your feedback ensures the recipient can undergo self-reflection and retrospectives in their own time.

3. Write your feedback statements on cards, give it to the person physically — Writing down your feedback on cards during the session and handing it directly to the person, or people, solidifies it and ensures they can’t simply forget or dismiss the feedback provided.

4. No interruptions or responding to feedback for 24 hrs — Provide feedback directly and ensure that no one interrupts, gets defensive or attempts to justify their behaviour for at least 24 hours. The idea of this “post feedback cooling off” period is to allow individuals to consider it for what it is and contemplate how they can improve. You should ensure that everyone understands they aren’t meant to interrupt or justify during the initial meeting.

5. Respond with ways to actionable improvement steps, don’t justify — After 24 hours, you should meet again with the individuals that provided feedback. Ideally, as a recipient of feedback, you should seek to provide measurable steps to improve on each component of feedback. Justifying your actions isn’t the purpose of supportive confrontation, it’s about setting actionable methodologies across a set time horizon so others can observe your behaviour and measure this improvement (or not).

6. Be Thankful — Thank all those who are involved in giving you feedback. The very purpose of this methodology is to grow and improve as an individual and to remember that those you mutually respect are telling you this because they care about you enough to want you to grow and be better as an individual. As the old adage suggests — “It’s better to know the devil you do, than the devil you don’t”

Feedback, in its rawest sense, is about listening, watching, learning and working to improve after receiving it. You should appreciate those with the courage to undertaken supportive confrontation with you enough, to ensure that you take the feedback seriously and find ways to action and improve on it as an individual. If you feel the feedback is overtly personal or hurtful in nature, you should have the confidence to speak to the individual, or group, after reflection 24 hours later and let them know that. Equally, you should seek to understand their reasoning nature of why they provided it — it will be better for all of you.

That way, you can become a better person, colleague, team member and leader as you grow.