Unpacking eBay’s New SQL Authoring Tool for Data Creation and Management
Discover how eBay built a tool for data modeling, ETL and SQL editing to enhance our analytics platform.
Gavin is a developer for eBay's unified data discovery, data analytics and data development platform, as well as our new open metadata solution. Gavin also manages marketing data products to support eBay's marketing business.
Automatic workflow generation is described. One or more files containing code statements for accessing and modifying information in a destination database is received. The code statements are parsed from the one or more files and dependencies between the code statements are determined. A dependency graph is built by arranging the code statements according to the dependencies between the code statements. The dependency graph is partitioned by identifying at least one barrier code statement having an unclear dependency and dividing the dependency graph between code statements occurring prior to the at least one barrier code statement and code statements occurring after the at least one barrier code statement. Jobs are scheduled based on the partitioned dependency graph, and the code statements are annotated according to the scheduled jobs. A workflow is then automatically generated based on the annotated code statements.