In the previous part of this article series, we explored the pitfalls of centralized AI and how web 3 could alleviate them. However, running models on-chain is impossible without paying extremely high gas fees. Attempting to increase the computation power of the underlying blockchain would increase node requirements for validators, which could reduce decentralization, since small, home validators would struggle.
In the following sections, we’ll go through some of the popular tools and techniques that are necessary for the further development of AI with Web3, namely, ZKPs, FHE and TEEs. We then explore the benefits of running AI models in a TEE-based coprocessor like Oyster and benchmark its performance against popular zkML libraries.
Zero Knowledge Proofs (ZKPs) are particularly important for AI and Web3 as they improve scaling and privacy. They allow for computations to be performed off-chain and then verified on-chain (verified compute), which is much more efficient than re-executing computations on all the nodes of a blockchain, thereby alleviating the load on the network and supporting more complex operations. zkML can, thus, enable AI models to operate in an on-chain environment. This ensures that the output from these off-chain computations is both trustworthy and verified.
Additionally, zkML can verify specific aspects of machine learning processes, such as confirming that a particular model was used for making predictions or that a model was trained on a specific dataset. zkML also can be used to verify computational processes. For example, it allows compute providers to demonstrate, via verifiable proof, that they have processed data using the correct model. This is especially relevant for developers who rely on decentralized compute providers (such as Akash) and want assurance about the accuracy and integrity of the computations.
zkML is also useful for users who need to run models on their data but wish to keep their data private. They can execute the model on their data, generate a proof, and subsequently verify the correct model's usage without compromising data privacy.
As discussed previously, fully homomorphic encryption (FHE) allows for computations to be performed directly on encrypted data without the necessity to decrypt it first. This technology has significant applications in the field of AI, particularly in the context of machine learning and the handling of sensitive data.
One of the primary applications of FHE is in the training of machine learning models using encrypted data sets. This approach ensures that the data remains encrypted and secure throughout the entire training process. As a result, FHE provides a comprehensive security solution that maintains data privacy from the beginning to the end of the machine learning pipeline. This is especially crucial in edge networks, where data security and privacy are paramount, and the computational resources are typically more limited compared to centralized data centers.
The utilization of FHE allows for the development of AI systems that preserve user privacy while still leveraging the advanced capabilities of AI. By ensuring that the data remains encrypted during both storage and processing, FHE offers robust cryptographic assurances against unauthorized access and data breaches. This is particularly relevant in scenarios where sensitive information is being processed, such as personal data in healthcare applications or confidential financial records.
FHE extends its utility to the operation of large language models in cloud environments. By enabling these models to process encrypted data, FHE ensures that user privacy and sensitive information are protected. This capability is increasingly important as more AI applications are deployed in cloud environments, where data security is a major concern. The ability to run models securely on encrypted data enhances the applicability of AI in fields that require strict confidentiality, such as legal, healthcare, and finance sectors.
FHE addresses the critical need to protect against potential information leaks and unauthorized access to sensitive data. In sectors where data privacy is not just a preference but a regulatory requirement, FHE provides a way to leverage AI's power without compromising on data security and compliance standards.
Trusted Execution Environments (TEEs) come with significant advantages when it comes to training and performing AI inference, in particular in terms of security assurances, isolation and data privacy and protection. As TEEs act as secure enclaves, they provide robust security and integrity for data and computations.
The first major benefit is enhanced security assurance. TEEs are specifically designed to counter vulnerabilities in systems with extensive Trusted Computing Bases (TCBs), which include the OS kernel, device drivers, and libraries. These components, due to their large attack surface, are more susceptible to exploits. TEEs, by offering a secure execution environment, protect critical applications even if the host operating system is compromised, maintaining the integrity and confidentiality of software within the enclave.
Another key advantage is isolation. Within enclaves, code and data are securely stored, and access is restricted solely to the code within the enclave. This design prevents external accesses, including those from other virtual machines or the hypervisor, safeguarding against physical attacks and threats from other virtual machines.
TEEs provide remote attestations which facilitate the process of verifying that the software is executed inside a genuine TEE. This feature is crucial for ensuring the authenticity and integrity of the software running within the enclave. It enables the establishment of trust between remote entities and the TEE, assuring that the software and its execution environment are secure and have not been tampered with.
Lastly, TEEs excel in data protection. The hardware-implemented security properties of a TEE safeguard the confidentiality and integrity of computations. This includes secure provisioning of code and data, such as cryptographic keys, into the enclave. TEEs also establish trusted communication channels for retrieving computing results and outputs, ensuring data remains secure throughout its lifecycle within the enclave. These features make TEEs an ideal environment for training AI and performing AI inference, particularly in applications requiring high levels of security and data integrity.
Marlin Oyster is an open platform for developers to deploy custom computation tasks or services over TEEs. Similar to Intel’s SGX and AWS Nitro Enclaves, through Oyster, developers can execute code in isolation, and ensure that neither the host nor any other application in it can alter the integrity of computations within the TEE. Apart from the computational integrity and confidentiality guarantees offered by TEEs, the Oyster platform provides additional benefits:
1. Uptime: Oyster ensures application availability through a monitoring protocol that penalizes nodes for downtime and reassigns tasks to operational nodes. This mechanism guarantees developers deploying on Oyster continuous app functionality and liveness for their end-users.
2. Serverless: Similar to AWS Lambda, Oyster's serverless framework allows developers to deploy applications without dedicating to a specific node rental. Developers benefit from cost savings and reduced management overhead by paying only for the runtime of their applications.
3. Networking: Oyster enclaves come pre-equipped with networking capabilities, facilitating secure TLS connections within the enclave. This feature enables the execution of external API queries and the operation of services with exposed endpoints, enhancing application integration with the internet.
4. Relay: Oyster supports the offloading of computationally intensive tasks to off-chain environments through relay contracts. These smart contracts enable the execution of functions on Oyster, ensuring reliable outcomes and event-based responses, thus optimizing on-chain resource usage.
In the benchmarking comparison between zkML frameworks and teeML (Oyster), the performance metrics suggest that Oyster operates with greater efficiency. Specifically, the Oyster framework demonstrates significantly lower total computation times across all machine learning models tested.
For the ordinary least squares model on Iris data, the zkML framework (RisQ) required over 32 seconds for proving and verification, whereas Oyster completed the task in just 0.047 seconds. Similarly, for the neural network on the same dataset, zkML (EZKL framework) had a total time of over 212 seconds for 500 inputs, in contrast to Oyster's 0.045 seconds. This substantial difference in processing time indicates that Oyster is vastly more efficient in these instances.
The LeNet models on MNIST data further solidify this observation. EZKL's zkML framework had proving and verification times amounting to 60 seconds, while Oyster required only 0.056 seconds. Even DDKang's zkML framework, which performed better than EZKL with a total time of approximately 3.33 seconds, was still outperformed by Oyster's 0.056 seconds.
Overall, the data reflects that Oyster offers a more efficient solution for machine learning tasks compared to the zkML frameworks tested. Its quicker computation times suggest that for the benchmarks provided, Oyster can handle the same tasks with significantly less processing time, making it preferable in terms of efficiency and speed.
For the widespread adoption of verifiable, decentralized AI, off-chain cryptographic verification systems must evolve beyond executing simple tasks like ordinary least squares calculations. The critical advancement required is the capability to process more complex tasks, specifically, to efficiently run prompts through popular LLMs. This necessitates enhancements in computational power, algorithm efficiency, and scalability of these systems to handle the sophisticated and resource-intensive demands of modern LLMs, thereby enabling more complex and diverse AI applications within a decentralized framework. zkML frameworks are still in their infancy and at this current stage, their ability to process these prompts is heavily impaired due to the computationally intensive task for generating zk proofs. It is hoped that zk proof marketplaces like Kalypso make zkML more affordable over time.
Although there have yet to be demonstrations of zkML protocols processing prompts for LLMs, it is reasonable to assume that the difference in processing times between Oyster's Trusted TEE and these zkML frameworks is at least as significant as in the examples previously discussed. Utilizing Marlin's Oyster, benchmark results for various LLMs can be established:
Enclave config: 12cpu 28gb memory (c6a.4xlarge)
Prompt: Ethereum is the community-run technology
Result: 'Ethereum is the community-run technology that enables the internet to function. In the same way that Linux did for computing, Ethereum will empower the internet to function in the future.’
Time taken to generate output: 22.091819524765015 sec
Tokens per second: 1.6295624703815754
Enclave config: 16cpu 30gb memory (c6a.8xlarge)
Prompt: Ethereum is the community-run technology
Result: 'Ethereum is the community-run technology which enables the decentralized application or smart contract to be built. The platform operates with no central authority or CEO. Instead, people working together issue and use smart contracts: apps that run independently on a large number of'
Time taken to generate output: 11.616417407989502 sec
Tokens per second: 4.304253044971607
The development and distribution of AI technologies are increasingly dominated by a select group of major corporations, possessing advanced hardware and sophisticated models. This concentration has raised concerns regarding censorship, inherent biases, and the challenge of verifying the integrity and fairness of AI systems. In contrast, the foundational principles of crypto—namely, permissionlessness and resistance to censorship—offer a pathway to democratize access to AI technologies.
The decentralized and open-source nature of blockchain technology enables a competitive landscape where decentralized AI can rival its centralized counterparts. This is facilitated through mechanisms such as DePINs, cryptographic proofs, and the use of public-private key pairs, which collectively ensure secure, transparent, and equitable AI development and usage. For decentralized AI to achieve its full potential, especially within blockchain ecosystems, it necessitates a robust infrastructure for offchain computation. This is critical for processing complex AI tasks efficiently, accurately, and verifiably.
Currently, TEEs emerge as the most viable solution for this requirement. TEEs provide a secure and isolated execution space for code, safeguarding the confidentiality and integrity of the data being processed. This makes them an optimal choice for offchain computations necessary for AI applications on the blockchain. As the field evolves, advancement of technologies like zkML, FHE and the enhancement of TEEs will be crucial for the decentralized AI ecosystem to overcome current limitations. This progress will facilitate a more open, accessible, and secure AI landscape, aligning with the decentralization ethos of the crypto community.
Follow our official social media channels to get the latest updates as and when they come out!
Twitter | Telegram Announcements | Telegram Chat | Discord | Website
Subscribe to our newsletter.
