Open-source orchestrator for data, AI and pipelines
Explore 🔮 Instill Core, the open-source orchestrator comprising a collection of source-available projects designed to streamline every aspect of building versatile AI features with unstructured data.
💧 Instill VDP: Unstructured data, AI and pipeline orchestration
Build, test and share your pipelines - No code required!
Instill VDP, also known as VDP (Versatile Data Pipeline), serves as a powerful data orchestrator tailored to address unstructured data ETL challenges.
- Simplify the journey of processing unstructured data from diverse sources, including AI applications, cloud/on-prem storage, and IoT devices. Utilize AI models and embed business logic to transform raw data into meaningful insights and actionable formats. Efficiently load processed data to warehouses, applications, or other destinations.
- Build flexible, durable pipelines with features like failover, automatic retries, rate limiting, and batching to handle high-volume data straight out of the box.
⚗️ Instill Model: Scalable MLOps and LLMOps for open-source or custom AI models
Instill Model, or simply Model, an advanced ModelOps/LLMOps platform focused on empowering users to seamlessly import, serve, fine-tune, and monitor Machine Learning (ML) models for continuous optimization.
💾 Instill Artifact: Unified unstructured data management platform
Instill Artifact, or simply Artifact, is your platform for transforming documents (e.g., HTML, PDF, CSV, PPTX, DOC), images (e.g., JPG, PNG, TIFF), audio (e.g., WAV, MP3 ) and video (e.g., MP4, MOV) of various formats into a unified AI-ready format. It ensures your data is clean, curated, and ready for extracting insights and building your knowledge base.
Not quite into self-hosting? We've got you covered with ☁️ Instill Cloud. It's a fully managed public cloud service, providing you with access to all the fantastic features of unstructured data orchestration without the burden of infrastructure management.
-
macOS or Linux - Instill Core works on macOS or Linux, but does not support Windows yet.
-
Docker and Docker Compose - Instill Core uses Docker Compose (specifically,
Compose V2andCompose specification) to run all services locally. Please install the latest stable Docker and Docker Compose before using Instill Core.
Use stable release version
Execute the following commands to pull pre-built images with all the dependencies to launch:
$ git clone -b v0.29.0-beta https://github.com/instill-ai/instill-core.git && cd instill-core
# Launch all services
$ make allNote
We have restructured our project repositories. If you need to access Instill Core projects up to version v0.13.0-beta, please refer to the instill-ai/deprecated-core repository.
Use the latest version for local development
Execute the following commands to build images with all the dependencies to launch:
$ git clone https://github.com/instill-ai/instill-core.git && cd instill-core
# Launch all services
$ make latest PROFILE=allImportant
Code in the main branch tracks under-development progress towards the next release and may not work as expected. If you are looking for a stable alpha version, please use latest release.
🚀 That's it! Once all the services are up with health status, the UI is ready to go at http://localhost:3000. Please find the default login credentials in the documentation.
To shut down all running services:
$ make down
Explore the documentation to discover all available deployment options.
To dive into Instill Core and Instill Cloud, you have a few options:
Note To access Instill Cloud, register an account with your email address.
⛅ Instill Console: Dive in Instill Core/Cloud with no coding
Instill Console, or Console is a user-friendly web-based UI application that improves accessibility and usability across both Instill Core and Instill Cloud. It allows you to dive into the creation of AI apps or the processing of unstructured data without the need for coding skills.
To access the Instill Core console, please launch Instill Core and navigate to http://localhost:3000. For the Instill Cloud console, simply go to https://instill.tech.
📺 Instill CLI: Bring Instill Core/Cloud to your command line
Instill CLI enables you to access Instill Core and Instill Cloud from your terminal. It can be installed by brew install instill-ai/tap/inst for Linux and macOS. To set up and get started with Instill CLI, head over here.
📦 Instill SDKs: Integrate Instill Core/Cloud with your language of choice
Instill SDKs make it easy for developers to integrate and interact with Instill Core and Cloud.
- Python SDK
- TypeScript SDK
- Stay tuned, as more SDKs are on the way!
For comprehensive guidance and resources, explore our documentation website and delve into our API reference.
Please refer to the Contributing Guidelines for more details.
We strongly believe in the power of community collaboration and deeply value your contributions. Head over to our Community repository, the central hub for discussing our open-source projects, raising issues, and sharing your brilliant ideas.
See the LICENSE file for licensing information.
![@dependabot[bot]](https://avatars.githubusercontent.com/in/29110?s=64&v=4){kind=small}
