Original LinkedIn Post
Infrastructure as Code for Tableau Prep: An Experiment with Python and AI-Driven ETL
Adam Mico Annabelle Rincon If AI can understand business requirements, why can't it build our ETL flows for us? I have been exploring the concept of programmable workflows, similar to the architecture found in tools like n8n where flows can be defined and extended via code. This inspired me to question why Tableau Prep could not have a similar programmable layer. To bridge this gap, I developed a Python SDK. It allows for the programmatic generation of Tableau Prep flow files (.tfl) and automatic publishing to Tableau Server. Deconstructing the Screenshot: For a clearer understanding of the attached image: The Right Side (The Logic): This shows the VS Code environment where the flow is defined. core/builder py & packager py: These serve as the foundation of the SDK. The Builder constructs the node logic, while the Packager compiles the definitions into the specific XML structure required by Tableau Prep. test_sdk_ultimate_v2 py: This is the user script. It demonstrates how inputs (SQL) and logic (Joins) are defined purely through code. output/ Directory: Upon execution, the valid .tfl file is automatically generated here, ready for deployment. The Left Side (The Result): The flow visible in the Tableau Prep interface?including the SQL inputs, the Join logic, and the Server Output?was generated entirely by the Python script. There was zero manual interaction. No dragging or dropping was involved; the code simply executed, and the flow was created. Current Progress & Roadmap: While the demo highlights a successful Join and Publish workflow, I am currently testing advanced data engineering features to ensure production readiness: Aggregations: Programmatically defining summary levels. Calculated Fields: Injecting complex business logic and formulas via script. Unions: Automatically merging multiple datasets from different sources. The Vision: The ultimate goal of this SDK is to enable AI agents to act as data engineers. In the future, an LLM could interpret a natural language business request, utilize this SDK to generate a complete .tfl file with necessary cleaning and calculations, and auto-publish it to the Server. This shifts the paradigm from manual GUI interaction to semantic-driven automation. I welcome your thoughts on this code-first approach to BI data preparation.
#Tableau #TableauPrep #Python #n8n #DataEngineering #AI #Automation #ETL #LowCode #Innovation #TechDemo #datafam #prep