Production-grade Data and ML Workflows Made Easy#
Highly scalable and flexible workflow orchestration for prototyping and production
Flyte is an open-source, Kubernetes-native workflow orchestrator implemented in Go. It enables highly concurrent, scalable and reproducible workflows for data processing, machine learning and analytics.
Created at Lyft in collaboration with Spotify, Freenome, and many others, Flyte provides first-class support for Python, Java, and Scala. Data Scientists and ML Engineers in the industry use Flyte to create:
ETL pipelines for petabyte-scale data processing.
Analytics workflows for business and finance use cases.
Machine learning pipelines for logistics, image processing, and cancer diagnostics.
Explore Flyte#
Get a birds-eye view 🦅 of Flyte at the official website:
From strongly typed interfaces to container-native DAGs, Flyte mitigates the trade-off between scalability and usability.
From strongly typed interfaces to container-native DAGs, Flyte mitigates the trade-off between scalability and usability.
Say goodbye to Airflow versioning pain and stepping over your teammate’s toes when you change your package versions. Ouch!
Unintuitive Python DSL boilerplate got you down? With flytekit you just
write Python code and Flyte compiles down to type-safe execution graphs.
Learn more about orchestration, Flyte, and everything in between.
Keep up-to-date with Flyte’s upcoming talks, conferences, and more.
Learn Flyte#
The following main sections in the documentation will guide you through your Flyte journey, whether you want to write Flyte workflows, deploy the Flyte platform to your K8s cluster, or extend and contribute its architecture and design.
Get your first workflow running, learn about the Flyte development lifecycle, and see the core use cases that Flyte enables.
A comprehensive view of Flyte’s functionality for data scientists, ML engineers, data engineers, and data analysts.
End-to-end examples of Flyte for data/feature engineering, machine learning, bioinformatics, and more.
Learn how to leverage a rich ecosystem of third-party tools and libraries to make your Flyte workflows even more effective.
Guides for platform engineers to deploy and maintain a Flyte cluster on your own infrastructure.
Reference for all of Flyte’s component libraries.
Dive deep into all of Flyte’s concepts, from tasks and workflows to the underlying Flyte scheduler.
Join the fast-growing Flyte community to get help, ask questions, and contribute!
Get Help#
Have questions or need support? The best way to reach us is through Slack:
Ask anything related to Flyte and get a response within a few hours.
Tell us about yourself. We’d love to know about you and what brings you to Flyte.
Share any suggestions or feedback you have on how to make Flyte better.
If you need any help with Flyte deployment, hit us up.