[SPARK-16294][SQL] Labelling support for the include_example Jekyll plugin

[liancheng]

What changes were proposed in this pull request?

This PR adds labelling support for the include_example Jekyll plugin, so that we may split a single source file into multiple line blocks with different labels, and include them in multiple code snippets in the generated HTML page.

How was this patch tested?

Manually tested.

[liancheng]

@mengxr Could you please help review this PR? Thanks!

[liancheng]

Also cc @yhuai and @rxin.

The background here is that I'm going to extract snippets from actual Scala/Java/Python/R source files rather than hard-code them in the SQL programming guide, so that they are easier to maintain.

[SparkQA]

Test build #61467 has finished for PR 13972 at commit 93f3398.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[liancheng]

@yinxusen Could you please help review this one since you're the original author of this plugin?

[yinxusen]

I'll take a look.

[yinxusen]

One small matter is that we can't use intersecting labels, e.g.

// $example on:init_session$
some code
// $example on:build_session$
some code
// $example off:init_session$
// $example off:build_session$

But this is a rare case I think. Otherwise LGTM.

[liancheng]

@yinxusen Thanks for the review! Also discussed with @mengxr. IIUC, overlapped labels are most useful for handling imports, since sometimes we may want to include one import line in multiple example snippet blocks. Regarding to this issue, I have two concerns:

1. It's a rare case, and we possibly can workaround this case while authoring documentations. Thus I guess we can do it in a follow-up PR.

2. Instead of overlapped labels, I'd prefer a multi-label approach:

   // $example on:A$
   import org.examples.A
   // $example off:A$
   // $example on:A:B$
   import org.examples.common._
   // $example off:A:B$
   // $example on:B$
   import org.examples.B
   // $example off:B$

   This approach achieves the same goal, and is much simpler to implement than handling overlapped scoping.

[liancheng]

@yinxusen @mengxr Actually I found that implementing overlapped labelling is far more easier than I expected earlier... So did it in the last commit.

Made the following experiment to illustrate the effect:

# $SPARK_HOME/examples/src/main/python/test.py

# $example on:foo$
import used.by.foo
# $example on:bar$
import common.stuff
# $example off:foo$
import used.by.bar
# $example off:bar$

Liquid template:

{% include_example foo python/test.py %}

{% include_example bar python/test.py %}

Screenshot:

[yinxusen]

LGTM

[liancheng]

@yinxusen Thanks!

[yinxusen]

@mengxr With this PR merged, I think we can also fix the SPARK-13015 (mllib-data-types.md ) with a consolidated example file.

[SparkQA]

Test build #61519 has finished for PR 13972 at commit 7ea9c75.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[mengxr]

LGTM2. Merged into master and branch-2.0. Thanks!

[mengxr]

@yinxusen Do you have time to consolidate example files for mllib-data-types.md?

[liancheng]

Thanks for the review!

[liancheng]

Other example snippets in the SQL programming guide will be updated in follow-up PRs.

[yinxusen]

@mengxr Sure