Kestra: Orchestrate Data, AI, and Infra Workflows
kestra-io/kestraKestra is an open-source orchestration and scheduling platform. You define workflows as declarative YAML — with a UI to help — and it runs them event-driven or on a schedule across data pipelines, AI tasks, and infrastructure jobs.
Reach for Kestra if you're outgrowing cron and want data, AI, and infra pipelines defined as versioned YAML with a real scheduler and UI behind them. Skip it if you just need to run a couple of scripts on a timer — a JVM orchestration platform is a lot of machinery for that.
The problem it solves
Once you have more than a handful of scheduled jobs, cron and glue scripts stop scaling: there's no shared view of what ran, no retries or dependencies, and no way to trigger a pipeline from an event instead of a fixed time. Teams end up hand-rolling an orchestration layer they'd rather not maintain.
What is it?
Kestra is an open-source (Apache-2.0), event-driven orchestration and scheduling platform written in Java. Workflows are declared in YAML (pipeline-as-code) and can be built or edited from the UI, then run on a schedule or triggered by events. It targets data engineering, AI, and infrastructure automation, with a plugin ecosystem for connecting to outside systems.
Why it's getting attention
Orchestration has usually meant Python-first tools; Kestra's pitch is a language-agnostic, YAML-declared platform that spans data, AI, and infra in one place. Its README carries a Trendshift badge and leans on high-availability and infrastructure-as-code framing, which lands as more teams push AI and data jobs into production and need a scheduler that isn't glued together by hand.
Key features
- ✓Declarative YAML workflows (pipeline-as-code) you can also build from the UI
- ✓Event-driven triggers plus scheduling, not just fixed-time cron
- ✓A plugin ecosystem for connecting to databases, cloud services, and other systems
- ✓Language-agnostic tasks, so pipelines aren't tied to one runtime
- ✓Aimed at data engineering, AI, and infrastructure automation in one platform
- ✓Self-hostable with a Docker quick-start and high-availability framing
Best use cases
- •Orchestrating ETL and data-engineering pipelines with dependencies and retries
- •Scheduling and chaining AI or batch jobs that outgrew cron
- •Triggering infrastructure automation from events rather than fixed times
- •Giving a team one UI and one source of truth for what runs and when
How to install / try
The README documents a Docker quick-start: `docker run --pull=always -it -p 8080:8080 --user=root --name kestra --restart=always -v kestra_data:/app/storage -v /var/run/docker.sock:/var/run/docker.sock ...`, which brings up the UI on port 8080. For production, see the project's deployment docs; Kestra is a JVM (Java) application.
How to use
Once it's running, you author workflows in YAML — either in the editor or the UI — defining tasks, triggers, and schedules. The README notes you can start from a quick-start flow directly in the UI in a few lines of YAML. Add plugins to connect to the databases, clouds, and services your pipelines depend on.
Strengths
- ✓Workflows are versionable YAML, so pipelines live in git like the rest of your code
- ✓Event-driven triggers go beyond fixed-time cron scheduling
- ✓Language-agnostic tasks avoid locking your pipelines to one runtime
- ✓One platform and UI covers data, AI, and infra jobs instead of separate tools
Limitations & risks
- △It's a JVM application, so expect a heavier memory/resource footprint than a lightweight scheduler
- △Defining everything in YAML has a learning curve and can get verbose for complex flows
- △Self-hosting a highly-available setup is real operational work, not a one-click deploy
- △For a couple of simple timed scripts, a full orchestration platform is overkill
Alternatives
Who should try it — and who should skip
Data, platform, and infra teams that have outgrown cron and want pipelines defined as versioned YAML with a scheduler, UI, and plugins behind them. If you're comfortable running a JVM service and want one place for data, AI, and infra workflows, it fits. For a few simple timed jobs, it's more than you need.
Frequently asked questions
As declarative YAML (pipeline-as-code). You can write them in the editor or build and edit them from the UI, then run them on a schedule or via event triggers.
Yes. The core is open source under the Apache-2.0 license. It's a Java (JVM) application you can self-host.
Airflow defines workflows in Python; Kestra uses declarative YAML and is language-agnostic for tasks, with built-in event triggers alongside scheduling. Which fits depends on whether your team prefers code-defined or YAML-defined pipelines.
The README provides a single Docker command that starts Kestra with its UI on port 8080, where you can create a quick-start flow in a few lines of YAML.