spotify/luigi
github.com/spotify/luigi
Repository
spotify/luigi
Luigi is a Python module that helps you build complex pipelines of batch jobs. It handles dependency resolution, workflow management, visualization etc. It also comes with Hadoop support built in.
From the page
Saved with the link. Notes from 2007 to 2016 are mostly the page's own words, so this is probably an excerpt.
Luigi is a Python module that helps you build complex pipelines of batch jobs. It handles dependency resolution, workflow management, visualization etc. It also comes with Hadoop support built in.