Introducing Jarvys - Spencer Sharp

I first watched Iron Man back in 2008. The Iron Man suit blew my mind. However, one of the tools used by Tony Stark particularly stood out to me. This was J.A.R.V.I.S. In the movies, Tony Stark would directly tell Jarvis to do something like "run a simulation and update me on the structural and compositional analysis of this element" and Jarvis would just do it. Ever since I have always aspired for an assistant like this on my computer that would just understand what I wanted it to do. Given the recent advancement in Large Language Models and specifically the release of GPT-3 by OpenAI, my cofounder Rohan and I, were hoping for someone to build an assistant like Jarvis. No one did. So we just built it ourselves.

Introducing Jarvys - an AI work assistant that allows you to directly tell your computer to do a task, rather than doing it by hand. Just give it a high level request and Jarvys does the rest.

We belive that Natural Language Interfaces are the future of computing. AI models are now good enough to understand our language and it's time to use them to interact with computers. Such interfaces can be powerfull as they speed up tasks that would otherwise require a lot of clicks and navigation. This will 10X our productivity and free up time for things we truly care about.

How does it work?

Jarvys uses GPT-3 by OpenAI, the same model behind ChatGPT. It is currently a Chrome Extension that can see what is happening in your browser and take actions like clicking, navigating and typing. It is generalized to work on any website and it's action space is the UI elements available on the page.

Future

A lot more work is left to be done. We want to implement better prompt chaining, better rendering of the DOM, embeddings and reduce latency (the gifs/videos you see have been sped up). We are getting feedback from our early test users for all the actions that Jarvys takes in order to implement Reinforcement Learning from Human Feedback (RLHF). We hope that this, along with improvements in the foundation models themselves will make Jarvys a truly universal AI work assitant capable of accompishing any task.