baomidou/jobs

Apache Airflow is a Python-based platform for programmatically authoring, scheduling, and monitoring workflows. It defines workflows as code (DAGs), which makes them maintainable, versionable, testable, and collaborative. The platform features a scheduler that executes tasks on an array of workers, following specified dependencies, complemented by command-line utilities and a user interface for visualizing pipelines, monitoring progress, and troubleshooting. Airflow is designed for workflows that are largely static and change slowly, finding use in traditional data pipelines, machine learning orchestration (training, retraining, evaluation, deployment), and coordinating steps in AI pipelines. It operates on principles of dynamic, extensible, and flexible pipeline definitions, utilizing Jinja templating, and recommends idempotent tasks that pass metadata via XCom rather than large data volumes directly.

🎓 Path to a free self-taught education in Computer Science!

Fair-code workflow automation platform with native AI capabilities. Combine visual building with custom code, self-host or cloud, 400+ integrations.

📝 Algorithms and data structures implemented in JavaScript with explanations and links to further readings