Skip to content
forked from Texera/texera

Big Data Analytics Using Interactive Workflows

Notifications You must be signed in to change notification settings

ethanwong16/texera

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Texera

Texera is a system to support collaborative, ML-centric data analytics as a cloud-based service using GUI-based workflows. It supports scalable computation with a parallel backend engine, and enables advanced AI/ML techniques. "Collaboration" is a key focus, and we want to enable an experience similar to existing services such as Google Docs, but for data analytics, especially for people with different backgrounds, including IT developers and domain scientists with limited programming background.

Motivation

  • Many data analysts need to spend a significant amount of effort on low-level computation to do data wrangling and preparation, and want to use latest AI/ML techniques. These tasks are especially tough for non-IT users.

  • Many workflow-based analysis systems are not parallel, making them not capable of dealing with big data sets.

  • Cloud-based services and technologies have emerged and advanced significantly in the past decade. Emerging browser-based techniques make it possible to develop powerful browser-based interfaces, which also benefit from high-speed networks.

  • Existing big data systems support little interaction during the execution of a long running job, making them hard to manage once they are started.

Goals

  • Provide data analytics as cloud services;
  • Provide a browser-based GUI to form a workflow without writing code;
  • Allow non-IT people to do data analytics;
  • Support collaborative data analytics;
  • Allow users to interact with the execution of a job;
  • Support huge volumes of data efficiently.

Sample Workflow

The following is a workflow formulated using the Texera GUI in a Web browser, which consists of operators such as regex search, sentiment analysis, user-defined function (UDF) in Python, and visualization.

Sample Texera Workflow

Publications:

  • (4/2017) A Demonstration of TextDB: Declarative and Scalable Text Analytics on Large Data Sets, Zuozhi Wang, Flavio Bayer, Seungjin Lee, Kishore Narendran, Xuxi Pan, Qing Tang, Jimmy Wang, Chen Li, ICDE 2017, Best Demo award, PDF, Video.
  • (1/2020) Amber: A Debuggable Dataflow system based on the Actor Model, Avinash Kumar, Zuozhi Wang, Shengquan Ni, Chen Li, VLDB 2020 PDF, Video, Slides
  • (7/2020) Demonstration of Interactive Runtime Debugging of Distributed Dataflows in Texera, Zuozhi Wang, Avinash Kumar, Shengquan Ni, Chen Li, VLDB 2020 PDF, Video, Slides
  • (4/2022) Optimizing Machine Learning Inference Queries with Correlative Proxy Models, Zhihui Yang, Zuozhi Wang, Yicong Huang, Yao Lu, Chen Li, X. Sean Wang, to appear in VLDB 2022.
  • (6/2022) Demonstration of Collaborative and Interactive Workflow-Based Data Analytics in Texera, Xiaozhen Liu, Zuozhi Wang, Shengquan Ni, Sadeem Alsudais, Yicong Huang, Avinash Kumar, Chen Li, to appear in VLDB 2022.
  • (6/2022) Demonstration of Accelerating Machine Learning Inference Queries with Correlative Proxy Models, Zhihui Yang, Yicong Huang, Zuozhi Wang, Feng Gao, Yao Lu, Chen Li, X. Sean Wang, to appear in VLDB 2022.
  • (7/2022) Drove: Tracking Execution Results of Workflows on Large Datasets, Sadeem Alsudais, to appear in the PhD workshop at VLDB 2022.

Videos

Getting Started

Texera was formally known as "TextDB" before August 28, 2017.

Acknowledgements

This project is supported by the National Science Foundation under the awards III 1745673, III 2107150, AWS Research Credits, and Google Cloud Platform Education Programs.

  • Yourkit Yourkit has given an open source license to use their profiler in this project.

About

Big Data Analytics Using Interactive Workflows

Resources

Stars

Watchers

Forks

Packages

No packages published

Languages

  • Java 31.6%
  • Scala 29.8%
  • TypeScript 21.0%
  • HTML 12.7%
  • Python 3.4%
  • SCSS 0.7%
  • Other 0.8%