Allows you to define complex dependency tree with ease for all your projects. You define packages and how they should be merged in a YAML file then the module generates corresponding gulp tasks for you.
Install using node :
npm install gulp-yaml-packages
Add the module to your gulpfile.js :
var yamlPackages = require('gulp-yaml-packages');
If you want to run the example, download the whole repository and do:
$ bower install
to install vendor dependencies.$ gulp
to execute the gulpfile.
--watch
- To watch for changes on files declared on the .yml and rerun tasks automatically.--env
- Set the environment (if prod, scripts and styles will be uglified, if dev sourcemaps will be generated).--theme
- Theme name that can be used by packages with dynamic theme.--strict
- Check that all 'standalone' packages define an output for each input.--verbose
- To get more feedback on what is happening.
Instead of manually writing tasks managing your resources, you define packages in a YAML configuration file. The configuration file will be far more readable when your project uses a lot of resources with a complex dependency tree.
The YAML is then used to generate gulp tasks that you can execute in your gulpfile. Here an example on how to use it:
With Gulp 3
:
// gulpfile.js
var gulp = require('gulp');
var loader = require('gulp-yaml-packages');
var tasks = loader.load(__dirname+'/app/config/gulp_packages.yml', gulp);
gulp.task('default', tasks);
With Gulp 4
:
// gulpfile.js
var gulp = require('gulp');
var loader = require('gulp-yaml-packages');
var tasks = loader.loadForGulp4(__dirname+'/app/config/app.packages.yml', gulp);
gulp.task('default', gulp.series(
gulp.series.call(gulp, tasks.series),
tasks.parallel.length ? gulp.parallel.call(gulp, tasks.parallel) : []
));
So first gulp
is required, then gulp-yaml-packages
.
The load
method can take 3 arguments :
path
- Theabsolute
path to the YAML file to load. It's very important to give an absolute path here.gulp
- Thegulp
instance created in your gulpfile.js.processors
- (Optional) An object defining custom processors. You can define new processors or override internal ones. More info in the processors section.
The load
method returns an array containing the name of tasks generated by the module.
This array is then passed as dependencies to the default
gulp task so they are executed.
In Gulp 4
an object containing two arrays is returned. It contains the following keys:
- series: for tasks that should be run in series
- parallel: for tasks that should be run in parallel (watch tasks)
You can still create your own tasks if you so desire, like:
With Gulp 3
:
// gulpfile.js
var gulp = require('gulp');
var loader = require('gulp-yaml-packages');
var tasks = loader.load(__dirname+'/app/config/gulp_packages.yml', gulp);
gulp.task('myCustomTask', function() {
// Do some other work.
});
gulp.task('default', ['myCustomTask'].concat(tasks));
With Gulp 4
:
var gulp = require('gulp');
var loader = require('gulp-yaml-packages');
var tasks = loader.loadForGulp4(__dirname+'/app/config/gulp_packages.yml', gulp);
gulp.task('myCustomTask', function(cb) {
// Do some other work.
cb();
});
gulp.task('default', gulp.series(
gulp.series.call(gulp, ['myCustomTask'].concat(tasks.series)),
tasks.parallel.length ? gulp.parallel.call(gulp, tasks.parallel) : []
));
The YAML is divided in 4 main parts :
packages
- This is where you define your resources and what should be done with them.processors
- Defines custom processing that should be done on resources. Each processor is simply a function that will be executed when the concerned resource is processed.parameters
- Inspired by Symfony parameters, allows you to define reusable strings.imports
- Allows you to import other YAML files into the current one.
A basic package file is defined like this :
packages:
# This is the package name
app:
# This is the type of resource
styles:
input: 'assets/less/my-private-file.less'
output: 'public/css/this-one-is-public.css'
3 types of resources are recognized :
styles
- For css, less, sass, etc.scripts
- For js, ts, coffee, etc.misc
- For everything that is not a script or a style (images, pdf, etc.).
The difference is that styles
and scripts
are merged in a single output and uglified in production.
Also, a sourcemap is generated in development so you can keep debuggable resources while testing.
misc
files are only copied if you don't set any processor on them.
You can set glob patterns as input :
packages:
app:
styles:
input: 'assets/less/**/*.less'
output: 'public/css/this-one-is-public.css'
In this case, every .less
file in the assets/less
folder will be compiled in this-one-is-public.css
.
When dealing with misc
resources (like images
), the structure of your folders after the first "globstar" (/**/
) will be preserved.
Consider the following example :
packages:
app:
misc:
input: 'assets/images/**/*'
output: 'public/images'
With a folder structure like this :
|-- assets
| |-- images
| | |-- sub
| | | |-- other.png
| | |-- cake.jpg
The output will be :
|-- public
| |-- images
| | |-- sub
| | | |-- other.png
| | |-- cake.jpg
You can set multiple input selectors for one output :
packages:
app:
styles:
input:
- 'assets/less/**/*.less'
- 'assets/sass/**/*.scss'
- 'assets/css/specific.css'
output: 'public/css/everything-in-here.css'
Packages can depend on one another, use the deps
key to define dependencies :
packages:
# jQuery
jquery:
# Here the 'input' is implicit. You can also give an array of paths.
scripts: 'vendor/jquery/jquery.js'
# Application
app:
# You can of course define an array of dependencies ([jquery, bootstrap, ...]).
deps: jquery
styles:
...
A package can define multiple versions. There are two criteria that define a version:
version
- The version number of the package (1.2, 3.2.3, etc.). Only support digits for now (3.1b will throw an error).theme
- The package's theme name. Can be anything alphanumerical.
For example, to define two versions of jQuery:
packages:
jquery:
-
version: 1.12.4
scripts: 'vendor/jquery-legacy/jquery.js'
-
version: 3.0.0
scripts: 'vendor/jquery/jquery.js'
To define two different themes:
packages:
bootstrap:
-
theme: default
styles: 'vendor/bootstrap/less/bootstrap.less'
-
theme: green
deps: 'bootstrap:default'
styles: 'assets/vendor/bootstrap/green-theme.less'
This will merge bootstrap.less
and green-theme.less
together in a single output.
Please note the bootstrap:default
in deps
. This is how you specify a theme in a dependency.
When importing a dependency, you can then ask for a minimum version:
packages:
app:
deps: 'jquery#1.9'
This will look for a jquery
package with a version >= 1.9
. The closest one will be selected.
You can combine theme
and version
like this:
packages:
app:
deps: 'bootstrap:green#3.3'
This will look for a bootstrap
package with a green
theme and a version >= 3.3
.
You can also require a package from another YAML file:
# app/config/gulp_packages.yml
packages:
app:
deps: 'app/config/vendor_packages.yml#jquery'
styles:
...
# app/config/vendor_packages.yml
packages:
jquery:
scripts: 'vendor/jquery/jquery.js'
Here the jquey
package defined in app/config/vendor_packages.yml
will be imported as dependency in the app
package.
You can also use theme
and version
filters when importing a dependency:
packages:
app:
deps: 'app/config/vendor_packages.yml#bootstrap:green#3.3.6'
styles:
...
By default, all files defined in your packages are automatically watched when the --watch
option is set.
But, for example, less
or sass
files may import
other files that are not defined in the yaml file, and so, not watched.
To solve this problem, you can add a watch
key to your styles
or scripts
definitions:
packages:
app:
styles:
watch: 'vendor/bootstrap/less/**/*.less'
input: 'vendor/bootstrap/less/bootstrap.less'
Now a change on any less
file in the vendor/bootstrap/less
folder will trigger the watcher.
But only bootstrap.less
will be compiled and copied.
Inspired by Symfony, parameters are a very basic replace by key thing. You define them like this :
parameters:
assets_dir: 'app/Resources/assets'
vendor_dir: 'app/Resources/vendor'
packages:
...
And use them like this :
packages:
app:
styles:
input: '%assets_dir%/styles/css/**/*.css'
output: 'web/css/frontend.css'
The %assets_dir%
will be replaced by app/Resources/assets
.
You can also use parameters in parameters:
parameters:
root_dir: '../../'
assets_dir: '%root_dir%/app/Resources/assets'
vendor_dir: '%root_dir%/app/Resources/vendor'
They are two parameters that are always set internally:
_theme
- The value is given by the--theme
option. If the option is not set, the default value isdefault
._env
- The value is given by the--env
option. If the option is not set, the default value isdev
.
Note: You can't override these values.
These parameters are useful to dynamically change a package theme for example.
Consider the following example:
packages:
bootstrap:
-
# Default theme (only styles for example). If no 'theme' key is defined, the value 'default' is set.
styles: 'vendor/bootstrap/less/bootstrap.less'
-
# 'default' theme is implicit.
deps: bootstrap
theme: red
styles: 'assets/vendor/bootstrap/less/my-RED-theme.less'
-
deps: bootstrap:default
theme: green
styles: 'assets/vendor/bootstrap/less/my-GREEN-theme.less'
# And your application that uses that
app:
deps: bootstrap:red
styles:
output: 'web/css/output.css'
Here 2 themes are defined for bootstrap, a red
one and a green
one. The application then uses the red
theme.
Not very exiting, but if you change the application dependency to :
app:
deps: bootstrap:%_theme%
styles:
output: 'web/css/output.css'
It becomes more interesting as the _theme
parameter is set by the --theme
option.
So you can run :
$ gulp --theme=green
to switch to the green theme in no time.
Processors are a set of callbacks you can use to make some custom processing on files.
Here the list of built-in processors:
typescript
- Compiles typescript files. Uses thegulp-typescript
module (GitHub).coffee
- Compiles coffee script files. Uses thegulp-coffee
module (GitHub).sass
- Compiles sass files. Uses thegulp-sass
module (GitHub).less
- Compiles less files. Uses thegulp-less
module (GitHub).cssurladjuster
- Rewrite urls in css files. Uses thegulp-css-url-adjuster
module (GitHub).image
- Optimize images. Uses thegulp-image-optimization
module (GitHub).
You can assign a processor to a package using the key processors
:
packages:
app:
styles:
output: 'public/css/public-file.css'
input:
files:
- 'assets/less/first.less'
- 'assets/less/second.less'
processors: less
You can define custom options like this:
packages:
app:
styles:
output: 'public/css/public-file.css'
input:
files: 'assets/less/styles.less'
processors:
less:
option1: val1
option2: val2
Of course it's very annoying to type this for every package using a less file. So you can a global configuration using the global key processors
:
#
# Defines that the 'sass' processor must be run
# on every file having a 'sass' or 'scss' extension.
#
processors:
-
name: sass
extensions: [sass, scss]
options:
option1: val1
#
# Then define packages like before
# Sass files will be automatically processed.
#
packages:
app:
styles:
output: 'public/css/public-file.css'
input:
- 'assets/less/first.scss'
- 'assets/less/second.scss'
Now both scss
files will use the sass
processor.
Notice that the processors
key is an array. It's very important as the array defines the order of execution.
Then, each processor configuration can define the following attributes:
name
- The name of your processor. That's the name you will refer to in your packages.callback
- (Optional) The name of the function callback to call when executing the processor. If not defined, takes the value ofname
.extensions
- (Optional) If defined, the processor will be automatically assigned to every file matching one of the extensions.options
- (Optional) Options object to send to the callback.
This configuration is already done internally for the built-in processors.
So any .scss
or .sass
file will be compiled using the sass
processor, same goes for less
, typescript
and coffee
.
You can very easily add your own logic by creating a custom processor. In your gulpfile.js
, simply add a third argument to the load
method:
// gulpfile.js
var gulp = require('gulp');
var loader = require('gulp-yaml-packages');
var tasks = loader.load(__dirname+'/app/config/gulp_packages.yml', gulp, {
myCustomProcessor: function(stream, options) {
// Do some custom processing here
//
// \!/ Be sure to return the stream or the pipeline will break.
// If you modify the stream, by doing a pipe for example, return the new one.
return stream;
}
});
gulp.task('default', tasks);
Then you can use it in any of your YAML files:
packages:
app:
scripts:
input:
files: 'assets/js/need-processing.js'
processors: myCustomProcessor
You can also define you want to apply it on every .js
file like this:
processors:
-
name: myCustomProcessor
extensions: js
options: ~
Note: Currently, the processor will only be applied to files defined in the YAML file defining the processor configuration. May have some improvement to do here, ideas are welcome.
To ensure processors are executed in a certain order, their configuration is defined as an array. They are then executed in the order they were defined in the array.
Built-in processors are executed in the following order:
typescript, coffee, sass, less, cssurladjuster, image
.
If you need to execute a custom processor before the sass
processor for example, you can do:
processors:
-
name: myCustomProcessor
options: ~
- sass
packages:
...
The sass
processor here is only a string. This means you don't define a new configuration, but simply indicate the order of execution.
The new order of execution will be:
typescript, coffee, myCustomProcessor, sass, less, cssurladjuster, image
.
Not defining the sass
element will result in the following order:
typescript, coffee, sass, less, cssurladjuster, image, myCustomProcessor
.
You can import other YAML files to merge their packages with the current file:
imports: '%vendor_dir%/vendor_packages.yml'
packages:
app:
# jQuery is defined in 'vendor_packages.yml'
deps: jquery
scripts: 'web/js/app.js'
You can import as many files as you want:
imports:
- app/config/file1.yml
- other/file2.yml
packages:
...
If packages with the same are found, they will be added as new versions. For example, writing :
# app/config/vendors1.yml
packages:
jquery:
version: 1.12
scripts: ...
# app/config/vendors2.yml
packages:
jquery:
version: 3.0.0
scripts: ...
# app/config/gulp_packages.yml
imports:
- app/config/vendors1.yml
- app/config/vendors2.yml
packages:
...
is the same as writing :
# app/config/gulp_packages.yml
packages:
jquery:
-
version: 1.12
scripts: ...
-
version: 3.0.0
scripts: ...
Note: processors and parameters are NOT merged. They are local to the YAML file defining them.
Here are explained some more "advanced" functionality that can be useful in certain cases but are not important enough to discuss in the main part of the documentation.
Packages' names are global by default. Defining packages with the same name in multiple files is the same as defining a package with multiple versions in a single file.
You can isolate a package name to its YAML file by adding the prefix @
:
# src/bundles/bundle1/config/bundle1_packages.yml
packages:
'@main':
styles:
...
# src/bundles/bundle2/config/bundle2_packages.yml
packages:
'@main':
styles:
...
Here we have 2 files defining a package named main
. Without the @
we would have two versions on the package available everywhere.
With the @
we can only include these packages by directly referring to the YAML file:
# app/config/gulp_packages.yml
packages:
app:
deps:
- 'src/bundles/bundle1/config/bundle1_packages.yml#main'
- 'src/bundles/bundle2/config/bundle2_packages.yml#main'
styles:
...
Note: when importing a YAML file using the imports
key, packages prefixed with @
are ignored.
Every package with input files and an output will create gulp tasks. But there are some cases where you may want to define a package without using it.
Imagine you create a vendor_packages.yml
that follows you between your projects.
In this file you would like to define packages for all the libraries you usually use:
# vendor_packages.yml
packages:
jquery:
scripts: 'vendor/jquery/jquery.js'
bootstrap:
styles: 'vendor/bootstrap/less/bootstrap.less'
scripts: 'vendor/bootstrap/js/bootstrap.js'
misc:
input: 'vendor/bootstrap/fonts/**'
output: 'web/fonts/bootstrap'
angular:
...
moment:
...
But in your current project, you don't use bootstrap
and jquery
, their files are not even present in your project.
For jquery it's all good, because you didn't define any output
in the package.
So until another package include its resources by doing deps: jquery
the package will not do anything.
But bootstrap
is different, it has misc
resources that must define an output
.
When the YAML file will be loaded, a gulp task will be generated and the fonts will be copied, event if none of your packages depend on bootstrap
.
You can prevent this behavior by adding a standalone
attribute to the package. It's value is true
by default, which means the package is enough to generate output on its own.
If you set it to false
, the package will only generate output when included in another package.
So :
# vendor_packages.yml
packages:
bootstrap:
standalone: false
styles: 'vendor/bootstrap/less/bootstrap.less'
scripts: 'vendor/bootstrap/js/bootstrap.js'
misc:
input: 'vendor/bootstrap/fonts/**'
output: 'web/fonts/bootstrap'
will only generate output if another package do deps: bootstrap
.
To go further on the theming topic, I would like to share a technique I use to easily theme libraries.
When you want to override styles of a library you have basically two solutions :
- Make a new stylesheet that will be loaded after the library one. In this file you'll override the parts you want to change.
- If your library is coded in
sass
orless
ou can recompile it after changing some variables.
When the first one is obvious, the second one can be trickier than it appear. I've came up with the following solution (depending on the language) :
With the following file in the role of the library :
// vendor/my-library/main.scss
$primary-color: #ff0000 !default;
.btn { background: $primary-color }
To modify the button color without rewriting the .btn
class or modifying the original file, you can simply create a new file :
// app/my-library-theme.scss
// First, override the variable you want to change
$primary-color: #00ff00;
// Then import the original library file
@import "../vendor/my-library/main.scss";
This will work because of the !default
attribute on the $primary-color
variable which indicates the variable must be set only if it doesn't exist yet.
In LESS it's even more powerful as you can set variables after they have been used.
// vendor/my-library/main.less
@primary-color: #ff0000;
.btn { background: @primary-color }
And the custom theme file :
// app/my-library-theme.less
// You can import the library BEFORE overriding its variables
@import "../vendor/my-library/main.less";
// This is legal, the previous import will use this value
@primary-color: #00ff00;
But by doing this, you're replacing a file of the library by one of your own (which then includes the library).
If you define your packages like I did in the Versions & themes section, you'll have a problem.
If you define your library like this :
my-library:
-
theme: default
styles: 'vendor/my-library/main.less'
-
deps: 'my-library:default'
theme: green
styles: 'app/my-library-theme.less'
When you include my-library:green
in a package, the library will be compiled and concatenated two times
because files of the green
and default
themes will be merged and because my-library-theme.less
do an @import
.
You could remove the deps
key and copy/paste common parts of the two packages, but its ugly and remove a lot of value to the module.
The easiest way I've found is to create a shared
theme to centralise what is common between themes :
my-library:
-
theme: shared
scripts: ...
misc: ...
-
deps: 'my-library:shared'
theme: default
styles: 'vendor/my-library/main.less'
-
deps: 'my-library:shared'
theme: green
styles: 'app/my-library-theme.less'
Like this you can do deps: 'my-library:green'
without having two copies of the styles.
As described in the Glob patterns section, you can define a glob pattern as input of a package:
packages:
app:
scripts:
input: 'assets/scripts/**/*.js'
output: 'public/js/this-one-is-public.js'
But in certain cases, you may be tempted to use a very vague glob like this:
packages:
app:
scripts:
input: 'assets/scripts/**'
output: 'public/js/this-one-is-public.js'
NEVER do this, because multiple problems can appear.
First, if other file than a script file is put in the scripts
directory, the sourcemap generator or uglifier may crash.
But even if you are 100% certain only scripts file will ever be in that directory, you may experience crashes anyway.
For example, if you use PHPStorm
or WebStorm
, temporary files may be created when the IDE is compiling your files.
For instance, when compiling a demo.ts
file, the IDE will create a demo.ts___jb_tmp___
file (probably to ensure no data are lost if an error occurs while compiling).
This file will only last for a very short time, but it's enough to trigger the watch, to start the task and include it file in the pipeline when resolving the glob. If you're unlucky, this file may be removed by the IDE before then end of the processing, and you'll get a fatal exception like :
events.js:141
throw er; // Unhandled 'error' event
^
Error: ENOENT: no such file or directory, stat '/app/Resources/assets/scripts/ts/demo.ts___jb_tmp___'
at Error (native)
To prevent this, always filter your globs by extensions. scripts/**/*.js
.
If you have multiple extensions, simply do: scripts/**/*.{ts,js,coffee}
.
Hope it helps.