Nicholas Demille is a professional writer, speaker, and storyteller and he has penned an exclusive byline which documents and charts the evolution of AI operating systems and how they will help to fully harness the transformative power of AI applications. 
To pick up a smartphone in 2022 is to use a sophisticated array of applications from thousands of developers around the world. That smartphones work as well as they do, is no small miracle.
Running in the background, apps gather and use vast amounts of data to get the work of our daily lives accomplished. The experience is so good, in fact, that it’s hard to fathom how much is happening behind the scenes, and how much the work of the operating system (OS) makes it all possible.
Operating systems are not a new concept. And even in the case of mobile computing, mobile operating systems have been in development since at least the 1990s. Without the seamless coordination of an operating system, it’s hard to imagine that smartphones would have gained the widespread adoption they enjoy today.
Personal Digital Assistants or PDAs fell short because, among other things, their apps were few and functioned miserably. Smartphones, with their increased computing power, improved internet connectivity, and powerful OS made everything work seamlessly, and the rest, as they say, is history.
So why then, has this approach not been followed in the AI development community?
Similar to mobile OS, AI operating systems (AI OS) are being developed for the efficient and effective development of seamless AI systems. Two advocates of this approach are faculty at MBZUAI and co-founders of AI startup Petuum — President Eric Xing and Assistant Professor Qirong Ho.
Xing, Ho, and an entire ecosystem of AI researchers and developers make up what is known as the CASL community, which is dedicated to solving these types of questions in an open-source capacity.
Open-source AI software
CASL, which stands for composable, automatic, and scalable learning, provides a “unified toolkit… …including distributed training, resource-adaptive scheduling, hyper-parameter tuning, and compositional model construction,” according to the community’s website.
Xing and Ho’s research and development is done within the larger context of their colleagues and collaborators from across the MBZUAI, CMU, Berkeley, Petuum, and Stanford campuses.
“With our AI OS, the question we set out to answer was: what is needed now and in the future for AI applications to work optimally,” Ho said. “Our AI OS connects software and hardware efficiently and ensures that AI software packages run as natively as possible.”
Out of this network of collaborations has come the production of the world’s first composable platform for MLOps (a portmanteau of “machine learning and DevOps), as well as an end-to-end AI OS, and quite a bit else.
In the case of MLOps, the results have been highly promising. According to a recent press release about the solution, users have seen improvements of 50% or more in time to value as well as in the efficiency of the ML team and resources. Further, it is projected that efficiencies should increase as AI projects get larger.
Developed in the classroom
Ho and his co-creators are constantly building and innovating on a stack of open source software platforms. CASL open-source software packages, which include AdaptDL, Tuun, Texar and more, are produced by the various members of the CASL community. Additionally, commercial software such as Rampart is produced by Ho, Xing and the wider team at Petuum, but is still built upon CASL open-source software.
One interesting example is Ho’s work on Rampart. This AI software package sprang from research done at MBZUAI, and has been developed, in part, in the university’s classroom environment. Rampart was developed to provide an MLOps interface and integration technology that lets developers include CASL software in their AI applications.
Rampart was also developed to function as a bridge between AI software and hardware. In Ho’s estimation, it is the most ideal environment yet developed for creating and using AI applications to their full potential.
Another example of open-source tools for building better ML products is Forte, which brings together the best practices of ML engineering via a data-centric perspective, such as modularization and standardization. ML systems require data operations such as cleansing, transformation, and anonymization, which tends to vary widely across different applications.
The Forte team, which spans the CASL community, MBZUAI and Petuum, developed a standardization scheme called Data Pack that provides core ML abstractions and extensibility for building standardized solutions. So while Forte provides routines like data lookup, and augmentation, the extensibility allows one to build reusable tools like anonymization, de-biasing and more.
“AI consumers expect speed, quality, reliability and trustworthiness from AI applications. The AI OS solution we’ve developed provides developers with the tools to create applications that make consumers happy, and that’s really the most important part,” Ho said. “At the end of the day, we are developing ways for AI consumers to implement solutions, innovate, and succeed at whatever their industry demands.”
