Feature: Adding new exploration UI

[fabianmenges]

Swivel, a new exploration interface for Superset

Swivel is a new exploration interface with a primary design goal of fast and
interactive data exploration.

It currently only supports the following visualizations:

• Table
• Line Chart
• Bar Chart
• Area Chart

The main features of Swivel are:

• Drag and Drop support for groupbys and filters
• Splitting query results into multiple charts
  
• Interactive range selection within charts
• Click to filter in the table view
• Full undo/redo history (ctrl/command + (shift) + z)
  
• State stored in local storage (multiple tabs with history)
• Running in auto and manual query mode
• Opening Swivel from existing explore view and vice versa.

Swivel state design

Swivel uses both redux-undo and redux-localstorage

Redux-undo allows you access to a full history of your local state.
This is specifically useful since its primary development goal was to allow
fast interactive exploration of datasources.

Redux-localstorage persist the state of your current state e.g. open new browser tabs that inherit
the state of the previous tab. This is useful if you are planning on drilling down into
specific parts of the data in parallel. It also brings you back to your last exploration state
after you close a tab or accidentally close your browser.

In order to efficiently use both redux-localstorage and redux-undo the following state
structure was chosen:

│
├── settings (undoable)
│   ├── query
│   └── viz
├── refData
└── control

• settings contains the part of the state that is both "undoable" and is
  persisted to local storage.
• query is the part of the state that contains all settings which require
  to run an updated query against the datasource.
• viz is the part of the state that contains all settings that only
  require a re-rendering of the visualization without the need to run a query.
• refData contains all reference data used by the Swivel. refData is not
  undoable nor is it persisted in local storage.
• control The control part of the state contains the global state/settings of Swivel
  that are not required for the visualization. This includes, whether a query is in progress,
  an error occoured, etc... the control state is not undoable but persisted in local storage.

Following the redux design guidelines there are reducers and actions for each of these partial states/stores.

Separation of query and viz state and Listeners

The query and the viz part of the settings state are separated to easily identify when a query needs to be rerun and when it is enough to rerender the the visualization.
For this purpose I've chosen to use renderless React components that located in the listeners folder. I could have directly subscribed to the redux state changes but this seemed cleaner.

These listeners will trigger the running of the updated query or rerendering the vizualization.

The viz state (with its reducers and actions) is extremely incomplete and I can potentially reuse big parts of the current explore view including the related react component architecture to complete it.

TODOs

• Write more unit tests
• Support Having filters
• Support more visualizations?
• More keyboard shortcuts

[coveralls]

Coverage decreased (-0.1%) to 68.987% when pulling baa8635af44def17e98921778a6fb1eb2f849a32 on tc-dc:fmenges/swivel_exploration into 3dfdde1 on apache:master.

[coveralls]

Coverage decreased (-0.1%) to 68.987% when pulling baa8635af44def17e98921778a6fb1eb2f849a32 on tc-dc:fmenges/swivel_exploration into 3dfdde1 on apache:master.

[coveralls]

Coverage decreased (-0.1%) to 68.987% when pulling 121cb27 on tc-dc:fmenges/swivel_exploration into 147c12d on apache:master.

[coveralls]

Coverage decreased (-0.1%) to 68.987% when pulling 121cb27 on tc-dc:fmenges/swivel_exploration into 147c12d on apache:master.

[coveralls]

Coverage decreased (-0.1%) to 68.987% when pulling ebfc4f3e68db7f105e7e381dd128a10c63355b9d on tc-dc:fmenges/swivel_exploration into 147c12d on apache:master.

[coveralls]

Coverage decreased (-0.1%) to 68.987% when pulling ebfc4f3e68db7f105e7e381dd128a10c63355b9d on tc-dc:fmenges/swivel_exploration into 147c12d on apache:master.

[coveralls]

Coverage decreased (-0.1%) to 68.987% when pulling ebfc4f3e68db7f105e7e381dd128a10c63355b9d on tc-dc:fmenges/swivel_exploration into 147c12d on apache:master.

[coveralls]

Coverage decreased (-0.1%) to 68.987% when pulling ebfc4f3e68db7f105e7e381dd128a10c63355b9d on tc-dc:fmenges/swivel_exploration into 147c12d on apache:master.

[coveralls]

Coverage decreased (-0.1%) to 68.982% when pulling 61f39c4754e07792224f9d7842fff11c2040d37c on tc-dc:fmenges/swivel_exploration into 7c1b56f on apache:master.

[xrmx]

Could you please remove some animated gifs? This crashed my firefox twice :)

[fabianmenges]

Worked on my machine :P ... I removed all but two, hope this fixes it.

[DaimonPl]

@fabianmenges this looks great, any plans when it will be ready? :)

[fabianmenges]

@DaimonPl We are using this in production already. I'll try to push an updated version of this tomorrow with bug fixes, a few more features and rebased to the current tip of master. Its currently not clear if this will make it upstream (into the master branch) anytime soon. I'll try to make it easier for people to check this version of superset out and run. The more community interest there is for this feature the more likely it is we'll get it into the master branch.

[rafalnowak]

@fabianmenges hi, any updates on updated version of this branch? This feature looks really great and we would like to give it a try.

[DaimonPl]

@fabianmenges @mistercrunch hi, I saw recent full annotation framework PR being merged. That's great!

Any chances to look into this feature as well? We are currently using external tool (pivot) for similar interaction.

It would be really, really great to have such feature built into superset. We are getting very, very good feedback from non-tech users for whom static dashboards sometimes are not enough.

[fabianmenges]

@DaimonPl just pushed an updated version. I'm updating the description, a lot of the 'todos' are now in. UI did not change much but quite a few new features around filters, sessions, request canceling and more.

There are discussions getting it into the master branch but nothing concrete. I built Swivel for the exact use case you are describing.

[qin4zhang]

That looks great.

[kalimuthu123]

yes this also looking great