Choose and implement a simple build/template process?

[snostorm]

Soon, the project will need more than 1-2 pages and a consistent look. With that we'll likely benefit from some sort of build process we can document for contributors in the project's ./README.md.

My recommendation is the builds (for now) happen locally with the static output committed in to the master repo, to support the build process already in place via #18. Later we could work with the build team to automate some things here.

If you take a look at https://www.staticgen.com/ there's a list of static-site generator tools we can pick from. Note, there is an option to filter it to only the JS-based projects (I couldn't hyperlink the filtered list directly.)

Some of the top projects by Github popularity/forks (with 2.1K - 3.8K stars each) include:

• http://hexo.io;
• http://brunch.io;
• http://metalsmith.io;
• http://harpjs.com; and
• http://assemble.io/

I need to refresh myself on these projects as it is been a couple of years since I've needed to chose one and the list has grown considerably, but overall I'd recommend us choosing a solution which offers a few simple page-template variations combined with markdown or a dead simple alternative for content.

A simple way of writing content (say in GitHub-flavoured markdown) will make it VERY easy for contributors to help us build guides, version release notes, open governance/contribution articles, etc.

Thoughts?

[snostorm]

Complication:

The current project structure assumes the root folder of the master branch is where our static files live. Thus, it gets awkward to mix-and-match our build tool / package.json / public files, etc. all in the root directory.

Options:

1. Use a gh-pages style approach whereby the master branch contains the source/tools (and possibly a compiled dist folder) and another branch contains the latest static assets to be served.
2. Coordinate with the build team to serve the website out of a sub-directory in the project, say dist or public. That way, Metalsmith/Hexo/whatever's configuration, as well as source markdown/HTML-partials/templates/etc. files could co-exist in the same branch.

I am leaning toward option 2 as it is the easiest to manage.

On a semi-related topic, it seems we've moved away from GitHub.io hosting although a copy of the site runs there too. Perhaps this repo should simply be called website or iojs.org?_

[Fishrock123]

This project will be renamed if we host it elsewhere. It only makes sense.

[Fishrock123]

I'm still leaning to a simple js file. Just needs a markdown parser and something like handlebars. I may be wrong here.

I guess we could use something like gulp of we want to also do stylesheets alongside that? I dunno.

@rvagg how should we pick up the build info, should this read it, or should we get it piped in via args or something?

cc also @zeke @mikeal

[rvagg]

@Fishrock123 perhaps if the build process was fed an argument or environment variable that gave a path to the /dist/ version of this? https://iojs.org/download/nightly/index.json it'll be on disk.

[snostorm]

It depends on if we want to have contributors version control the dist/ output or not. It certainly makes it easier to deploy and for people to clone and view the site locally if we do. (But, it is also an extra step to remember.)

Re: "simple script + parser", many of the existing npm libraries (some of which I linked to above) are slim and basically give you that functionality with minimal overhead. Maybe tomorrow I can whip up a proof of concept with Metalsmith** to show how light it would be.

If we start grouping content like blog/announcement, a guides section, etc. we'll soon be reinventing the wheel versus using a solution already maintained in the community.

Regardless we move over master to having some sort of generator process in place with template(s), it will be MUCH nicer to review/create content-related PRs.

** I like that Metalsmith is all mix-and-match plugins. Similar to using Grunt/Gulp in that sense. We can mix-and-match modules to suite our needs and adding our own custom logic/step (if needed down the line) would be simple enough.

[Fishrock123]

Metalsmith feels like a gulp that people won't be familiar with.

[yoshuawuyts]

I've recently built my fair share of static sites, and I'm -1 on both metalsmith and gulp. They seem to have a flourishing ecosystem, but both do plugin dependencies wrong and are restrictive once you need to deviate. My suggestion is that you guys just roll your own build process, which should be simple enough. Using marked coupled with browserify and myth should get you 90% there probably.

If you're going the way of running an actual server to spin up the files, I suggest taking a look at watchify-request and myth-request for simple cached asset endpoints.

[snostorm]

I'm not sure I'd agree with the gulp system being restrictive. It is pretty easy to add your own processing steps to deal with custom "things". Though my experience with gulp is more on the building/managing static resources for single-page applications or similar use cases. I've never tried to organize template/markdown/auto-linking/etc. through it.

Maintaining our own build system is just one more thing to maintain. Even with Gulp I feel like we'd have to do a lot of thinking about the setup. Something like metalsmith just gets us up and running very fast.

[therebelrobot]

I use gulp for almost everything. It's really simple to set up our own tasks, even for templating and stuff. I could get a good build system for this project running on gulp in about a half hour or less. The plugins for less/sass/myth/etc are brainless plugins, the community has built most of the plugins we would ever need for our build.

[Fishrock123]

@therebelrobot same.

[snostorm]

Alright, the more I think about this the more I'm sure all the plugins/examples we need probably do exist. +1 for Gulp, as I do love it in general.

I'm not sure if we want to generate on deploy, or build and commit to the repo. While I know the general "gut feeling" is to not include stuff that's generated, there can be benefits, like for users cloning the repo to quickly browse the site offline. It also vastly simplifies the deploy process, as the current rsync step is all that is needed.

If we are going to have the build team run our script we should probably also add a CI step to make sure nothing breaks mid-process. I can imagine something sneaking in to the pipe that suddenly breaks the site while building on the server and causes something crazy to happen as a result in production.

Okay, maybe this is my compromise: short term, we bundle the generated resources (as the team is small) and work toward shifting things over to a CI / build-team process that automates that?

Edit: I realize we also want to automate some download links / version numbers, and that requires at least something must happen as part of the build teams deployment processes to make this happen.

[therebelrobot]

I like the way that sounds. +1 @snostorm

[snostorm]

@therebelrobot I think this brings us back to #26 needing to come so we can start adding a Gulpfile, package.json, etc. to the project. Let's switch that part of the discussion there to finalize the name so we can carry on?

[snostorm]

See #84 relating to the build process HOSTING questions. Later we can have the github webhook there (which triggers a site deploy) also run any build process scripts we need.

Likely, we should be able to make it fairly generic, a shell script which always triggers git pull && npm install && gulp publish.

Once in place we wouldn't need to modify the build stuff very often as the site build code will all be in this repo.

Relating to short term goals, to save from having to coordinate with build before we lock down a process, I vote (as +1 a few times above) that we just commit and track the ./public directory in the short term. We can always remove those files once the build process steps are stable enough to outsource to the build server.

[therebelrobot]

So @Fishrock123, I've created the build/gulpjs branch, where you can find the stuff I've been working on. I found https://github.com/sindresorhus/gulp-markdown, which could be a good bet for converting markdown files during the build. IMO, we can just set these md files in separate language folders, and have gulp just loop through them and build static copies for every language, then we can get indexed by the googles properly, new translators can find their folder to work in quickly, and we can focus our efforts on more problematic issues. Let me know when you're online tonight and we can chat further about this.

[therebelrobot]

+1 @snostorm on the publishing th /public branch for now.

[therebelrobot]

Notes from meeting with @Fishrock123:

• gulp - stripped from https://github.com/greypants/gulp-starter
• image optimization should NOT be part of build step, but can be in the gulpfile for optimizing new commits
• templates in either handlebars or https://github.com/substack/html-template, Fishrock123 to play with this
• gulp-markdown for content conversion
• stylus for css (Fishrock123 to handle this)
• separate lang folders in content folder: /content/code-of-lang/file.md
• source folder, with source/templates source/style