In a rapidly evolving Generative AI landscape where many tech companies are tapping into OpenAI’s APIs, ZenML, a startup based in Munich, Germany, believes that the future of AI lies in small, in-house models. Their open-source framework aims to act as the glue that binds various AI tools together, facilitating collaboration among data scientists, machine-learning engineers, and platform engineers in the development of specialized AI models.
ZenML’s uniqueness stems from its empowering approach, enabling companies to construct their private, tailored AI models. While they might not aim to rival giants like GPT-4, they can craft smaller models optimized for their precise needs, thereby reducing their dependence on external API providers like OpenAI and Anthropic.
Louis Coppey, a partner at VC firm Point Nine, noted, “The idea is that, once the first wave of hype with everyone using OpenAI or closed-source APIs is over, [ZenML] will enable people to build their stack.”
Adam Probst and Hamza Tahir, the founders of ZenML, previously worked together on a company that was building ML pipelines for other companies in a specific industry. “Day in, day out, we needed to build machine learning models and bring machine learning into production,” ZenML CEO Adam Probst told me.
“We are connecting the open-source tools that are focusing on specific steps of the value chain to build a machine learning pipeline — everything on the back of the hyperscalers, so everything on the back of AWS and Google — and also on-prem solutions,” Probst said.
The main concept of ZenML is pipelines. When you write a pipeline, you can then run it locally or deploy it using open-source tools like Airflow or Kubeflow. You can also take advantage of managed cloud services, such as EC2, Vertex Pipelines and Sagemaker. ZenML also integrates with open-source ML tools from Hugging Face, MLflow, TensorFlow, PyTorch, etc.
“ZenML is sort of the thing that brings everything together into one single unified experience — it’s multi-vendor, multi-cloud,” ZenML CTO Hamza Tahir said. It brings connectors, observability and auditability to ML workflows.
Initially, ZenML released its framework as an open-source tool on GitHub, amassing more than 3,000 stars on the platform. They have now expanded to provide a cloud version with managed servers, with plans to introduce triggers for continuous integrations and deployment (CI/CD).
Several companies have already embraced ZenML for various use cases, including industrial applications, e-commerce recommendation systems, and medical image recognition. Notable clients include Rivian, Playtika, and Leroy Merlin.
As the AI ecosystem evolves, ZenML’s success hinges on its capacity to provide companies with more tailored and cost-effective AI solutions. While many businesses currently use APIs for AI features, these APIs can be overly sophisticated and expensive, primarily designed for general use cases, not specific ones.
Adam Probst, ZenML CEO, explained, “OpenAI, or these large language models built behind closed doors, are built for general use cases—not for specific use cases. So currently it’s way too trained and way too expensive for specific use cases.”
He added, “OpenAI will have a future, but we think the majority of the market will have to have its own solution. And this is why open source is very appealing to them.”
The CEO of OpenAI, Sam Altman, also acknowledges the need for both specialized and broad models in the AI landscape. Ethical and legal considerations in AI usage further underline the significance of specialized models, with European regulations encouraging companies to develop models trained on specific datasets in specific ways.
Hamza Tahir concluded, “The value of MLOps is that we believe that 99% of AI use cases will be driven by more specialized, cheaper, smaller models that will be trained in-house.”
As the AI landscape continues to evolve, ZenML’s innovative approach could pave the way for companies to harness the power of AI while maintaining control and reducing costs.
—————————————————————————————————
About Innovation Streets:
Welcome to InnovationStreets.com! Get the latest insights from the finance and technology industries. Discover industry updates on Fintech Innovations, Regulatory Developments, Industry Insights, Events, and Banks/Insurance partnerships with fintechs. A platform for C-level executives to submit Thought Leadership content for their peers. Subscribe for updates!
