Menu

Lineman.js

Lineman takes everything you love about building server-side applications so that you can find joy in your client-side applications

Lineman is a command-line utility that is hyper-focused on helping web developers build first-class JavaScript web applications. Lineman provides a thin wrapper around a number of client-side productivity tools (primarily Express, Grunt, and Testem), with the goal of helping developers focus on writing awesome web apps instead of worrying about workflow configuration.

lineman new

lineman run

lineman spec

lineman build

The project is hosted on GitHub. You can report bugs and discuss features on the issues page or send tweets to @linemanjs.

Lineman is open-source technology from Test Double, LLC.

Understanding Lineman

Building Happiness

Lineman's mission is to make fat-client JavasScript web applications as easy to build as traditional server-side HTML web applications. We accomplish this by establishing a narrow set of conventions and curating default configurations for your application's build tasks to make the developer's experience both predictable and convenient. Once you've scaled Lineman's gentle learning curve, you'll be able to ramp-up quickly on any project that uses Lineman.

For a broad overview of the motivations behind Lineman, consider watching this introductory presentation:

To get started using Lineman right away, continue here. Otherwise, read on.

What does 'modern-client' mean?

Traditional web applications are written using server-side frameworks that happen to support the generation of HTML and embedding of JavaScript and CSS. Many of these frameworks maintain strong opinions about how you should structure your server-side code yet often treat client-side code as an afterthought.

  // Traditional Web Application

  +-----------------------+                       +------------------------+
  |        Browser        |                       |         Server         |
  |-----------------------|     HTTP Request      |------------------------|
  | - Parse & Render:     |     (Form Data)       | - Render Templates     |
  | -- HTML               | --------------------> | - Routing (App & API)  |
  | -- CSS                |                       | - Security             |
  | -- JavaScript         |   "Full Page Reload"  | - Authentication       |
  |                       |                       | - Data Storage         |
  |                       | <-------------------- | - Application Logic    |
  |                       |     HTTP Response     | - Domain Logic         |
  |                       |    (HTML, JS, CSS)    | - Asset Management     |
  |                       |                       | - Static Asset Serving |
  +-----------------------+                       +------------------------+

With the rise of the MV* pattern on the client-side, many web developers are building web applications using a "modern-client" model that eschews the traditional "full-page reload" lifecycle for a leaner pattern. In this model, web-servers are responsible for data-delivery (usually as JSON), security, authentication and data-processing/storage; the client is responsible for html rendering, routing, templating, and some application logic.

  // Modern Web Application

  +-----------------------+                       +------------------------+
  |        Browser        |                       |         Server         |
  |-----------------------|     HTTP Request      |------------------------|
  | - Parse & Render:     |     (JSON Data)       |                        |
  | -- HTML               | --------------------> | - Routing (API)        |
  | -- CSS                |                       | - Security             |
  | -- JavaScript         |  "Partial Rendering"  | - Authentication       |
  | - Render Templates    |                       | - Data Storage         |
  | - Application Logic   | <-------------------- |                        |
  | - Routing (App)       |     HTTP Response     | - Domain Logic         |
  |                       |     (JSON Data)       |                        |
  |                       |                       | - Static Asset Serving |
  +-----------------------+                       +------------------------+

What is a 'first-class' web app?

It is an unfortunate reality that client-side code is often not given the same level of care and attention that is given to server-side code. Many times this "client-side inferiority complex" is aggravated as web developers move between server-side frameworks and have to re-learn how to manage client-side code within the opinions of that server-side framework.

A lack of consistency in the way client-side code is managed across server-side frameworks has left many web developers frustrated, disillusioned, or apathetic when it comes to caring about the quality of the client-side code that they write.

We built Lineman to craft a developer experience that would liberate our client-side assets from the opinions of each server-side framework and allow them to stand on their own as first-class citizens. At its core, Lineman is a fully realized client-side workflow based on sensible task defaults and a simple transparent proxy that allows web developers to completely decouple their client-side from the server-side. In addition, it produces happiness by building assets, mocking servers, and running specs on every file change.

This separation allows client-side assets to stand on their own as first-class citizens.

  // Development Cycle `lineman run`

  +-----------------------+                      +------------------------+
  |     Modern Client     |                      |        API Server      |
  |-----------------------|      apiProxy        |------------------------|
  | Lineman:              | <------------------> | Any Backend Platform:  |
  | - Any Client MV*      |                      | - Security             |
  | - Test Execution      |         OR           | - Auth                 |
  | - CI Integration      |                      | - API Routing          |
  | - Build Tool          | <------------------+ | - Application Logic    |
  | - Coffee, JavaScript  |                    | | - ORM                  |
  | - SASS, Less          |      apiStubbing   | |                        |
  | - Dev Server          |                    | |                        |
  | - pushState Simulator | +------------------+ |                        |
  +-----------------------+                      +------------------------+
  // Production Deployment `lineman build`

  +-----------------------+                      +---------------------------------------+
  |     Modern Client     |                      |    Production Application Server      |
  |-----------------------|   `lineman build`    +-----------------------+---------------|
  |                       |                      |                       |               |
  | Lineman:              | -------------------> | build artifact: +     | web front end:|
  | - Any Client MV*      |                      | ./dist          |     | - nginx/apache|
  | - Test Execution      |                      | - index.html    |     | - app server  |
  | - CI Integration      |                      | - app.css       +-------+ public/html |
  | - Build Tool          |                      | - app.js              |               |
  | - Coffee, JavaScript  |                      | - img/*               |               |
  | - SASS, Less          |                      |                       |               |
  | - Dev Server          |                      |                       |               |
  | - pushState Simulator |                      |                       |               |
  +-----------------------+                      +-----------------------+---------------+

How does Lineman work?

Task Lifecycle

Lineman is a thin abstraction layer on top of Grunt's task automation features. Lineman comes with a number of default tasks that have been pre-configured. These tasks are organized into lifecycle phases that identify at which point during the build each task should run. Those phases can be illustrated as a set of ordered tasks, like this:

appTasks:
  common: ["coffee", "less", "jshint", "handlebars", "jst", "concat", "images:dev", "webfonts:dev", "pages:dev"]
  dev:    ["server", "watch"]
  dist:   ["uglify", "cssmin", "images:dist", "webfonts:dist", "pages:dist"]

The "common" phase is run during most Lineman actions (e.g. during both lineman run & lineman build). The "dev" phase only runs during development tasks (e.g. lineman run). The "dist" phase only runs during production ("distribution") tasks (e.g. lineman build).

Task Configuration

Grunt tasks are configured using targets that define two critical pieces of information:

  1. what options a task will use
  2. what files a task will operate on

Lineman splits each of these concerns into two separate files:

  1. <your-application>/config/application{.js,.coffee}
  2. <your-application>/config/files{.js,.coffee}

Lineman allows you to override both its default task options and default file locations within your apps configuration.

// <your-application>/config/application.js

module.exports = function(lineman) {
  //Override application configuration here
  return {};
};

And:

// <your-application/config/files.js

module.exports = function(lineman) {
  //Override file patterns here
  return {};
};

Which Configuration Wins?

Given that Lineman intentionally splits task configuration between multiple locations you may be wondering the order in which configuration files are merged:

  1. <your-project>/config/application{.js,.coffee}
  2. <your-project>/node_modules/lineman/config/application.coffee
  3. /usr/local/lib/node_modules/lineman/config/application.coffee **

** If Lineman is not installed locally it will pull configuration from the global installation of Lineman

Working with Lineman

Project Templates

If you aren't sure what client-side framework you want to use for your app you may want to start with Lineman's default, framework-agnostic, project template. You can generate this project template from the command line:

$ lineman new your-project

Lineman then gives you some instructions on how to get up and running with this default template:


    _
   | |
   | |     _ _ __   ___ _ __ ___   __ _ _ __
   | |    | |  _ \ / _ \  _   _ \ / _  |  _ \
   | |____| | | | |  __/ | | | | | (_| | | | |
   |______|_|_| |_|____|_| |_| |_|___,_|_| |_|

- Assembling your new project directory in "/Users/davidmosher/code/your-project"
- Created a new project in "your-project/" with Lineman. Yay!

Getting started:
  1. "cd your-project" into your new project directory
  2. Start working on your project!
    * "lineman run" starts a web server at http://localhost:8000
    * "lineman build" bundles a distribution in the "dist" directory
    * "lineman clean" empties the "dist" and "generated" directories
    * "lineman spec" runs specs from the "specs" directory using testem

For more info, check out http://github.com/linemanjs/lineman

If you already know that you want to use a particular client-side framework (like Backbone, Ember, or Angular) we have a few framework-templates floating around to help you get up-and-running even more faster:

Project Directory Structure

Lineman generates a very particular directory structure. It looks like this:

.
├── app
│   ├── js                  # <-- JS & CoffeeScript
│   ├── img                 # <-- images (are merged into the 'img' folder inside of generated & dist)
│   ├── static              # <-- any other static files that need to be included in your built app
│   └── pages               # <-- static HTML pages (underscore and handlebars templates are supported)
│       └── index.us        # <-- a template used to produce the application's index.html
│   └── templates           # <-- client-side templates
│       ├── other.us        # <-- other templates will be compiled to a window.JST object
│       └── thing.hb        # <-- underscore & handlebars are both already set up
│       └── _partial.hb     # <-- a handlebars partial, usable from within other handlebars templates
├── config
│   ├── application.js      # <-- Override application configuration
│   ├── files.js            # <-- Override named file patterns
│   ├── server.js           # <-- Define custom server-side endpoints to aid in development
│   └── spec.json           # <-- Override spec run configurations
├── dist                    # <-- Generated, production-ready app assets
├── generated               # <-- Generated, pre-production app assets
├── grunt.js                # <-- gruntfile defines app's task config
├── package.json            # <-- Project's package.json
├── tasks                   # <-- Custom grunt tasks can be defined here
├── spec
│   ├── helpers             # <-- Spec helpers (loaded before other specs)
│   └── some-spec.coffee    # <-- All the Jasmine specs you can write (JS or Coffee)
└── vendor                  # <-- 3rd-party assets will be prepended or merged into the application
    ├── js                  # <-- 3rd-party Javascript
    │   └── underscore.js   # <-- Underscore, because underscore is fantastic.
    ├── img                 # <-- 3rd-party images (are merged into the 'img' folder inside of generated & dist)
    └── css                 # <-- 3rd-party CSS

Develop, Test, Build

Web Server (Express JS)

Once you've generated a directory structure with the default project template, or cloned a framework project template then it's time to get developing with Lineman.

In your command-line interface:

  1. Navigate to your project directory:

    $ cd your-project
  2. Start Lineman's development environment:

    $ lineman run
  3. Point your web-browser at http://localhost:8000

Test Runner (Testem)

Lineman integrates a full-featured test runner called Testem. Whether you are using the default project template, or have cloned a framework project template you will have a set of default tests within the spec directory. Lineman works by compiling all your files whenever it detects a change on disk during lineman run.

In another command-line interface session:

$ lineman spec

By default, Lineman configures Testem to re-run tests on every file change and automatically launches Chrome to execute tests using a generated Jasmine specrunner html file:

TEST'EM 'SCRIPTS!
Open the URL below in a browser to connect.
http://localhost:7357/
━━━━━━━━━━━━━━┓
  Chrome 31.0 ┃
    1/1 ✔     ┃
              ┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
✔ 1 tests complete.









[Press ENTER to run tests; q to quit]

Continuous Integration (Testem)

Lineman comes pre-configured to execute your tests using Testems "CI" mode which will run tests against Phantom JS and yield output in TAP13 format which is easy for CI environments (such as Jenkins) to consume:

$ lineman spec-ci

The spec-ci command will compile your app, and then run tests within PhantomJS 1.9:

Running "spec-ci" task
ok 1 PhantomJS 1.9 - .helloText then expect(this.result).toEqual("Hello, World!").

1..1
# tests 1
# pass  1
# fail  0

Done, without errors.

If you'd like to output your CI report to a file, you can configure this in your config/application.js like so:

'spec-ci': {
  options: {
    reporter: {
      type: 'xunit',
      dest: 'path/to/my/report.xml'
    }
  }
}

Other supported reporter types include "dot" and "tap".

Build Tool (Grunt)

When you are ready to bundle up your modern-client app and add the compiled JavaScript, HTML, CSS, Images, and Webfonts to your static web server you can use Lineman's build command:

$ lineman build

This will execute all of Lineman's common and dist phase tasks:

Running "common" task

Running "coffee:compile" (coffee) task
File generated/js/app.coffee.js created.
File generated/js/spec.coffee.js created.
>> Destination (generated/js/spec-helpers.coffee.js) not written because compiled files were empty.

Running "less:compile" (less) task
>> Destination not written because no source files were found.
File generated/css/app.less.css created.

Running "jshint:files" (jshint) task

Running "handlebars:compile" (handlebars) task
>> Destination not written because compiled files were empty.

Running "jst:compile" (jst) task
File "generated/template/underscore.js" created.

Running "concat:js" (concat) task
File "generated/js/app-b45ee785ae12b3e0ee816ecb7c31d4b9.js" created.

Running "concat:spec" (concat) task
File "generated/js/spec.js" created.

Running "concat:css" (concat) task
File "generated/css/app-388558805ea3fbaa55a57a117d34ab81.css" created.

Running "images:dev" (images) task
Copying images to 'generated/img'

Running "webfonts:dev" (webfonts) task
Copying webfonts to 'generated/webfonts'

Running "pages:dev" (pages) task
generated/index.html generated from app/pages/index.us

Running "dist" task

Running "uglify:js" (uglify) task
File "dist/js/app-b45ee785ae12b3e0ee816ecb7c31d4b9.js" created.

Running "cssmin:compress" (cssmin) task
File dist/css/app-388558805ea3fbaa55a57a117d34ab81.css created.

Running "images:dist" (images) task
Copying images to 'dist/img'

Running "webfonts:dist" (webfonts) task
Copying webfonts to 'dist/webfonts'

Running "pages:dist" (pages) task
dist/index.html generated from app/pages/index.us

Done, without errors.

Lineman will use Grunt to execute a number of tasks and generate your production bundle in the dist folder at the root of your project directory, here's what the output looks like:

$ tree dist

dist
├── css
│   └── app.css
├── favicon.ico
├── index.html
└── js
    └── app.js

Advanced Features

API Integration

Most web apps require some interaction with a server, and no developer could be expected to write working code without either faking the server-side or plugging the client and server together. Lineman offers support for both!

API Stubbing

Users may define custom HTTP services to aid development in config/server.js by exporting a function named drawRoutes. The provided app is an Express.js application, so you have the entirety of its API available to you.

Here's a trivial example:

module.exports = {
  drawRoutes: function(app) {
    app.get('/api/greeting/:message', function(req, res){
      res.json({ message: "OK, "+req.params.message });
    });
  }
};

API stubbing is a powerful feature that can be leveraged to use Lineman as a rapid prototyping tool and to enable front-end development to run a bit ahead of backend development (and importantly, to gather feedback and validation before investing in full-stack development). In the past, test double has used API stubbing to prove out a lightweight executable specification of the services a front-end application will need in order to work. In fact, it's often useful to keep a few data structures in memory to test more complex interactions (pushing objects onto an array, deleting them by ID, etc.).

Note that this feature is intentionally only available to the development server. It is not intended to build out an actual production server, nor is it intended to be used in Lineman's built in unit test suite run with lineman spec.

Lineman's also watching changes you make to the server stubbing, so there's typically no need to restart your development server upon changes.

API Proxying

Lineman also provides a facility to forward any requests that it doesn't know how to respond to a proxy service. Typically, if you're developing a client-side app in Lineman and intend to pair it to a server-side app (written, say, in Ruby on Rails), you could run a local Rails server on port 3000 while running Lineman on port 8000, and your JavaScript could seamlessly send requests to Rails on the same port as Lineman's development server.

To enable proxying, set the enabled flag on the apiProxy configuration of the server task in config/application.js, like this:

server: {
  apiProxy: {
    enabled: true,
    port: 3000
  }
}

HTML5 pushState Simulation

Lineman provides a way to direct requests to either your API or serve up generated/index.html in order to simulate what using HTML5 pushState with a configured static server will behave like. This requires both apiProxy.prefix and server.pushState configuration property to be set, so that the apiProxy correctly knows how route requests to the api you have configured. To enable pushState in your application set the pushState flag on the server task in config/application.js and the prefix on the apiProxy configuration of the server task, like this:

server: {
  // enables HTML5 pushState;
  // Lineman will serve `generated/index.html` for any request that does not match the apiProxy.prefix
  pushState: true,
  apiProxy: {
    enabled: true,
    port: 3000,
    prefix: 'api' // request paths that contain 'api' will now be the only ones forwarded to the apiProxy
  }
}

Lineman Pages

Lineman allows you to generate multiple HTML files using its "pages" feature. You could use this to create an app with separate pages, or customize Lineman's tasks to generate separate web apps for each HTML file. Any template files placed within app/pages will automatically be compiled to HTML and placed within the root of dist when built. Here's an example:

  1. Add an other-app.us template at app/pages/other-app.us
  2. During lineman run you can access this page at http://localhost:8000/other-app.html
  3. During lineman build this page will be placed at dist/other-app.html

Heroku Buildpack

Deploying your app to heroku couldn't be easier. Once you have the heroku toolbelt installed, simply run this from your project:

$ heroku create --stack cedar --buildpack http://github.com/linemanjs/heroku-buildpack-lineman.git

Now, whenever you git push heroku master, our custom buildpack will build your project with lineman and then start serving your minified site assets with apache!

What's really neat about this workflow is that while heroku takes care of building the assets for you (meaning you don't have to worry about checking in or transferring any generated assets), at runtime node is nowhere to be found! Your site is just static assets running on apache.

Using Sass

Lineman supports Sass via grunt-contrib-sass. Because the sass task requires Ruby & the "sass" gem to be installed in order to function, it is disabled by default.

First, to enable sass, in your config/application.js file, uncomment enableSass: true:

module.exports = function(lineman) {
  return {
    //...
    enableSass: true
    //...
  };
};

Once Sass is enabled, it will look for a "main.scss" or "main.sass" file in app/css. Rather than concatenating all the source files you include in your project, Sass files will only be included as you @import them from your main file. If you'd like to override this default entry-point file, override the files.sass.main config in config/files.js like so:

module.exports = function(lineman) {
  return {
    sass: {
      main:"app/css/my_app_styles.{sass,scss}"
    }
  };
};

Setting up ruby

Sass requires the "sass" ruby gem to be installed. (For info on installing ruby see rvm or rbenv+ruby-build). To install the "sass" gem, run:

$ gem install sass

We recommend you lock down the version of Sass your project uses with Bundler. To do so, create a Gemfile at your project root with:

source "https://rubygems.org"

gem "sass"

Then run bundle install, which will generate a Gemfile.lock (commit both to version control). Finally, tell the grunt task to invoke the locked version of Sass with the following configuration in config/application.js:

sass: {
  options: {
    bundleExec: true
  }
}

For more information on overriding the task's options, please check out grunt-contrib-sass's README.

Customizing Lineman

Configuring Lineman

Every lineman project comes with a handful of minimal configuration files out of the box. These are:

config/
├── application.js
├── files.js
├── lineman.js
├── server.js
└── spec.json

Any of the four JavaScript files can be changed to a CoffeeScript file, so if you prefer CoffeeScript we encourage you to first convert the file you're editing to CoffeeScript and to save it with a ".coffee" extension.

lineman config

Before we begin, let's look at a handy tool in our toolbox, the lineman config command.

At any time, you can ask Lineman what its configuration will look like as it will be passed to grunt (that is, after any plugin overrides or user overrides you've defined). To see the entire (very large) configuration object, just run:

$ lineman config

Which is a handy way to scan and search for the option(s) you're interested in overriding. If you want to narrow your search, just provide lineman config the property path you're interested in. If you provide:

$ lineman config jshint.options

You'll receive:

{
  "curly": true,
  "eqeqeq": true,
  "latedef": true,
  "newcap": true,
  "noarg": true,
  "boss": true,
  "eqnull": true,
  "sub": true,
  "browser": true
}

Note that this command can also be useful for script automation. For instance, the lineman heroku buildpack checks to see if Sass is enabled in a bash script by invoking lineman config enableSass and relying on a plaintext response of "true" or "false"

Now, let's discuss each of these files, one at a time.

application.js

The config/application file is your application's opportunity to override any project or task configuration defaults set by Lineman. The object you return from the function the module exports will be merged onto the default configuration just before being passed to grunt.config.init, which means your application can override, replace, or remove any of the built-in tasks, or define all new ones.

Let's start with an example. Suppose you want to override Lineman's built-in jshint configuration to allow constructor functions to start with lower-case letters. Currently, Lineman sets the property associated with this rule ("newcap") to true. You could relax this rule in config/application like this:

module.exports = function(lineman) {
  return {
    jshint: {
      options: {
        newcap: false
      }
    }
  };
};

We can verify this change by either running and testing our app or interrogating the configuration with lineman config jshint.options and verifying that we see "newcap": false where we expect it.

Sometimes, you'll want to peek at the existing lineman application config. You can do this from the function's provided lineman argument. Specifically, we frequently want to provide a concatenation—as opposed to an outright overwrite—of array-value properties.

In example, if we were to append a task to the "common" build phase (meaning we want it to run both during lineman run and lineman build), we'd want to concatenate our addition to the existing array to ensure we don't incidentally overwrite a previous configuration file or plugin's changes. [Editor's note: we agree that this is currently a bit onerous and error-prone; we're working on an API that improves on this particular case]:

module.exports = function(lineman) {
  var app = lineman.config.application;
  return {
    appendTasks: {
      common: app.appendTasks.common.concat("myTask")
    }
  };
};

(A word of warning: mutating values directly on the lineman.config.application is not supported and will probably break in a subsequent release.)

One last fun trick with config/application is that, you can refactor your application config—should it ever become so large as to be unwieldy—into as many "plugin" files as you like in your app under config/plugins/*.{js,coffee}. Despite the name, there's no need to pull that config out into a separate module, it's just a convenience for breaking up large configurations down into more focused ones. For details on the API for plugin files, check out Creating Lineman Plugins

files.js

To make it easier to reuse file patterns across multiple task configurations, Lineman (for the most part) pulls out file glob configuration into a separate section of the application config under a "files" property. It's recommended you override this from the config/files file.

The most common override folks need to make is to override the load order of the third-party vendor JavaScript files their app depends on. To discover the default file globs for JavaScript, one could run lineman config files.js and see:

{
  "app": "app/js/**/*.js",
  "vendor": "vendor/js/**/*.js",
  "spec": "spec/**/*.js",
  "specHelpers": "spec/helpers/**/*.js",
  "concatenated": "generated/js/app-b45ee785ae12b3e0ee816ecb7c31d4b9.js",
  "concatenatedSpec": "generated/js/spec.js",
  "minified": "dist/js/app-b45ee785ae12b3e0ee816ecb7c31d4b9.js",
  "minifiedWebRelative": "js/app-b45ee785ae12b3e0ee816ecb7c31d4b9.js"
}

Knowing this, we can easily override the "files.js.vendor" property by supplying an array of patterns, like so:

module.exports = function(lineman) {
  return {
    js: {
      vendor: [
        "vendor/js/underscore.js",
        "vendor/js/jquery.js",
        "vendor/js/bootstrap/**/*.js",
        "vendor/js/backbone.js",
        "vendor/js/**/*.js"
      ]
    }
  };
};

The above configuration will still load all of the scripts placed in vendor/js, but will ensure Underscore comes first, followed by jQuery, anything in the "vendor/js/bootstrap" directory, and then Backbone.

To understand the supported patterns (including how to exclude patterns), check out the underlying grunt.file.expand API.

If you intend to extend Lineman with custom tasks or plugins, we encourage you to add additional file property paths to the files configuration (as opposed to hard-coding them in your relevant task configurations). You can use Grunt's configuration interpolation to expand them later. For instance, the files.js.vendor path above is referenced from an actual task configuration like so:

$ lineman config concat_sourcemap.js
{
  "src": [
    "<%= files.js.vendor %>",
    "<%= files.template.generated %>",
    "<%= files.js.app %>",
    "<%= files.coffee.generated %>"
  ],
  "dest": "<%= files.js.concatenated %>"
}

lineman.js

This is a handy little shim that you can require whenever you need access to the application's lineman object from some Node.js context, for instance from a custom grunt task. Keep in mind that its lineman.config value won't be fully-baked when required this way.

server.js

You can use config/server to specify custom routes that Lineman's development server will respond to, for the purpose of API Stubbing.

spec.json

This file is the Testem (the underlying test runner invoked by lineman spec and lineman spec-ci) configuration file. Check out details for customizing it at Testem's readme. Some Lineman-specific examples are available here.

Adding Lineman Plugins

Lineman includes a plugin mechanism for packaging and sharing project configurations that are likely to be common across multiple Lineman projects.

Our goal with Lineman has always been to encourage commonality and to discourage duplication between web projects by encapsulating sensible defaults for grunt task configurations. But what about for task configurations that aren't common to everyone? For example, it obviously wouldn't make sense for application-framework-specific build settings to make their way into the default Lineman configuration.

For cases like these, we created a way to package customizations to Grunt task configurations and Lineman's build behavior without requiring any duplication or additional end-user configuration.

We have a handful of plugins already available:

As you can see by skimming the list above, the motivation behind various plugins can differ wildly. Some support productivity tools like bower and browserify. Some add tighter integration with application frameworks, like Angular or Ember. Others add entirely new languages, like dogescript & jade. We're also working on plugins that provide deploy-time behavior, like lineman-deploy-aws will.

If you're interested in a particular plugin, we recommend you poke around its repository a little bit to see what grunt task modules it includes, what custom tasks it defines, and what configurations it overrides or adds in config/plugins/*.js. When you're ready to give a plugin a spin, all you need to do is install and save it to your project's package.json, like so:

$ npm install --save-dev lineman-jade

The example above will install lineman-jade to node_modules/ and, merely by virtue of being listed as a dependency in your package.json, will be discovered by the lineman CLI, loaded, and applied to the project's configuration. You can add as many plugins to your project as you like; Lineman will start with its default configurations and successively merge in the configuration extensions defined by each plugin you specify, before ultimately merging in your application's configuration.

It's also worth noting that plugins can include plugins and Lineman will load them recursively. So for example, if you wanted to publish a slightly (but hopefully, meaningfully) different Angular configuration, you could write a plugin that depends on lineman-angular and only overrides the configuration to the extent that it differs from that plugin. On that note, let's talk about how to write your own plugin next.

Creating Lineman Plugins

Writing your own Lineman plugin is refreshingly easy! The most important thing to remember is that Lineman's job is typically only to provide sensible default task configuration and some level of build awareness. If a Grunt task module distributed via npm is a chocolate, then a Lineman plugin that sets it up and incorporates it into users' project builds is merely its thin candy shell. This narrow focus has enabled us to write clear and focused plugins that provide a lot of value without requiring a lot of code.

Here are some of the things a typical plugin might do, in descending order of likelihood:

  1. Depend on one or more grunt task modules, adding them to loadNpmTasks (Example)
  2. Define task configurations in a config/plugins/my-task-here.{js,coffee} file (Example)
  3. Prepend/Append the task to the dev, dist, or common build phase (Example, Doc)
  4. Define new file patterns to be used by your tasks & users (Example)
  5. Include all-new Grunt tasks within the plugin module in a tasks/*.{js,coffee} file
  6. Create prerequisite files in users' projects with a postinstall script (Example: pt. 1 pt. 2)

Project setup

First off, every plugin's name must start with the string "lineman-" in order to be discovered and auto-loaded by Lineman.

Next, the plugin's package.json file should declare "lineman" as a peer dependency like so:

"peerDependencies": {
  "lineman": ">= 0.24.0"
}

Try to ensure you pick a version specifier that's high enough to ensure it'll work (for reference, plugins were introduced in lineman@0.19.0).

Finally, any grunt task modules or other runtime dependencies you need should be included in the "dependencies" object, so that they'll be installed and available to the end user (this is a little counter-intuitive since your plugin itself will be in your users' "devDependencies" objects).

Defining configurations

The plugin API couldn't be much more familiar, as it closely resembles the format of each Lineman project's config/application and config/files files. Your plugin may define any number of JavaScript and CoffeeScript files under config/plugins and they'll be automatically picked up and run by Lineman, merged into an ever-growing projec.t configuration.

A plugin file looks like:

module.exports = function(lineman) {
  return {
    files: {
      //Any file patterns you have can go here
    },
    config: {
      //Any project & task configurations go here
    }
  };
};

The provided lineman object exposes the current configuration (as it exists just prior to loading your plugin file) under lineman.config. Keep in mind that you'll definitely want to concat any array-type configuration values unless you intend to overwrite them entirely. For instance, setting loadNpmTasks: ["my-task-module"] would have the unintended consequence of overwriting all the tasks loaded by any previous plugins.

Publish your plugin

We recommend testing your plugin locally with npm link from a local project, but once it's ready, all you need to do is publish it to npm. If this is your first time, check out npm's docs.

Adding Grunt Tasks

Adding NPM based tasks

To load NPM-based tasks that aren't part of the standard Lineman dependencies, you can add the module names to the loadNpmTasks object in config/application.js. Note that these still need to be added to your app's package.json dependencies and installed to node_modules with npm install.

  loadNpmTasks: lineman.config.application.loadNpmTasks.concat("npm_task_to_load")

Adding Custom tasks

Lineman will automatically require all files in the tasks directory and load them into Grunt. If you have custom tasks, you can leave them there and add them to the build as above.

Lineman can easily be extended to do extra grunt-work for your application above-and-beyond the built-in grunt tasks. You may add tasks to your project in two ways:

  • If you're writing a task yourself, add it to the tasks/ directory. Lineman will automatically load any tasks found here.
  • If you want to use a task that's packaged in an external npm module, add it to your package.json as a dependency and run npm install.

Once they're loaded, you can manually run the task from the command line using lineman grunt (which just delegates through to grunt):

$ lineman grunt taskname

But you're probably more interested in adding the custom task to run along with the other tasks in lineman run and/or lineman build. You can add any task to these commands by adding it to the appropriate array under the appendTasks object in config/application.js:

  prependTasks: {
    common: ["A"],
    dev: ["B"],
    dist: ["C"]
  },
  appendTasks: {
    common: ["D"],
    dev: ["E"],
    dist: ["F"]
  }

In the above example, tasks "A" & "D" would run during both lineman run and lineman build. Meanwhile, "B" & "E" would run only during lineman run, while "C" & "F" would only run during lineman build.

Tasks specified under prependTasks way will be run before Lineman's built-in tasks for the corresponding phase, while tasks specified under appendTasks will run immediately afterward. For reference, check out Lineman's default configuration.

If you need more fine-grained control—say you want to replace or remove a default task—you can use custom JavaScript in your application config file to edit the appropriate array directly; here's an example of removing a task from the Ember.js template.

Removing Grunt Tasks

If you need to remove a task that's built into Lineman, you can use the "removeTasks" configuration. For example, if you wanted to disable Lineman's CoffeeScript task, you could do this:

  removeTasks: {
    common: ["coffee"]
  }

Test Framework Integration

Jasmine

Lineman is preconfigured to use Jasmine. Lineman also ships with a few Jasmine helpers:

They are located in spec/helpers/, along with helper.js (which just adds some helpful aliases).

Mocha

  1. Remove the jasmine-* helpers in spec/helpers/
  2. Change the framework in config/spec.json:
    {
     "framework": "mocha"
     ...
    }
  3. Put your desired expectation/assertion library in spec/helpers/, e.g. expect.js.

Mocha+Chai

  1. Remove the jasmine-* helpers in spec/helpers/
  2. Change the framework in config/spec.json:
    {
     "framework": "mocha+chai"
     ...
    }
  3. Set Chai as your expectation library in spec/helpers/helper.js:
    var expect = chai.expect;

QUnit

  1. Remove the jasmine-* helpers in spec/helpers/
  2. Change the framework in config/spec.json:
    {
     "framework": "qunit"
     ...
    }

Questions & Answers

Lineman vs Yeoman

We are commonly asked, "How does Lineman differ from Yeoman?". Firstly, Lineman is a tool designed primarily to ease the creation of modern-client web applications. It is client-side framework agnostic, but opinionated about the workflow and tools developers use to build any modern-client apps. However, given there is some overlap between the two tools we thought it best to lay things out with a feature grid:

Feature Lineman Yeoman
Generators 1 builtin scaffold, clonable templates, no sub-generators No builtin scaffolds, installable generators, many sub-generators
Command Line Interface Wrapped utilities that are accessed via CLI ie: lineman grunt Unwrapped utilities that are accessed via their CLI ie: grunt
HTML5 pushState Simulator builtin and enabled via config/application -
API Stubbing Prototyping builtin and enabled via config/server -
API Proxy builtin and enabled via config/application -
Test Runner testem, configured for dev and ci mode with lineman spec
and lineman spec-ci
-
Grunt Task Lifecycle preconfigured phases: common, dev, dist, split into config/files and config/application generator specific
Dependency Graph intentionally shallow and lightweight generator specific, often deep and heavy
Cute ASCII Text Greeter - snazzy wizard based CLI
Package Management none builtin but supports various, including bower, via extensions bower
SourceMap Generation builtin, defaults to grunt-concat-sourcemap -
Dev Server lineman run, spins up express.js generator specific
Directory Structure common across project templates generator specific
License MIT BSD

Sub Generators

Why doesn't Lineman have sub-generators for its project templates?

Generated code, like fish, begins to smell after three days.

For the maintainers, it's yet another thing they need to keep in sync with the library-dependent code that's being generated; worse, if the library makes BC-breaking changes, the code generators need to become smart enough to generate one among multiple versions of the code (after detecting which version of the library the user is depending on).

For the users, it can certainly help overcome "blank page paralysis" and help them to get started, but the generated code often introduces other immediate issues caused by an uncertain understanding of what was just generated. We saw this with rails scaffold all the time back in the day—folks would absolutely love it in demos, but upon looking at the bizarre controller & view code that emerged, they felt less command and ownership over it than they would have if they had rolled their own. That is to say that generating a quick start to an activity typically doesn't cure "blank slate paralysis", it merely defers it—and worse, to a point where you already have the carrying cost of code to maintain.

There is a caveat, I think: a generator that produces only a very tiny amount of code (like a rails g migration generator) can absolutely prove handy when (a) it's very small and (b) the code that it's generating has some element that makes it a pain to do by hand. (In the case of Rails database migrations, that painful aspect is the timestamp needed in the migration's file name.)

But generally, code generation is very hard to implement well, and it's even harder when the code being generated depends on a library you don't own. It essentially guarantees that Yeoman's sub-generators can't be delivered in a quality, forward-compatible way. That team is hereby doomed to get a bunch a 3 a.m. issues filed on account of generating out-of-date code, brought on by the innocent March of Progress by whatever library the generator targets.

Wrapping Libraries

Why does Lineman wrap grunt, testem and other tools instead of letting users reference them directly?

The universe of people who understand Grunt, testem, and the myriad plugins that Lineman depends on is a relatively tiny set of humans (in the thousands). The audience that needs a much better front-end development workflow with minimal additional cognitive load is absolutely enormous (in the millions). Therefore, it doesn't make sense to bottleneck adoption of the much better development workflows that are afforded by Grunt (et. al) by first demanding as a prerequisite an understanding of what Lineman uses "under the hood." The user should be able to get some value out of Lineman without understanding how it works at all.

Additionally, by maintaining our own CLI and owning the top-level process, we can smooth out the edges of some of the libs we depend on in ways that aren't always possible by configuration alone. Better yet, we can swap out a dependency more-or-less invisibly with an upgrade to Lineman.

Finally, Lineman doesn't hide anything that it depends on, because we know that every single project that uses Lineman long enough will need to customize some aspect of its workflow eventually. That's why Lineman is obscenely extensible—literally anything Lineman configures (whether it's grunt tasks, conventional file patterns, or test runner configuration) can be overridden without any precognition on the part of Lineman's developers. This is one of the reasons it's been so easy for us to adapt Lineman for use with a variety of front-end frameworks with minimal additional code.

We view Lineman as providing just one (skinny!) layer of your front-end workflow, as opposed to a monolithic approach like Rails provided:

Aspect The Rails Way Our front-end approach
Application Framework Rails Any of the 20 current front-runners
Conventions and Default Config Rails Lineman
Build tasks Rails (via Rake) Grunt

Test Runner

Why does Lineman use Testem as its default test runner?

We like Testem because Lineman & Testem seem to share a core motivation: make development a delightful, joyful activity, a priority that's been sorely lacking from front-end web development up until very recently. Testem was incredibly easy to integrate into our default Lineman configuration, supports a variety of test library adapters and usage patterns for the benefit of users, and is otherwise unopinionated and unbiased.

It does what it says on the tin, and not a thing more.

It could be argued that it's not the only test runner in Node-land to do so, but we've yet to see it argued that anything else is objectively better.

Package Management

Why doesn't Lineman use Bower or any other Package Management tools?

For the same reason Lineman doesn't care to know which front-end JavaScript framework you choose to build your application with: if you ask a hundred front-end developers how it ought to work, you'll get a hundred different answers. Even developers who agree on a tool tend to arrive with different expectations at where to draw the boundaries between automation and convention. A bunch of questions arise, and the answer to each will narrow Lineman's audience; questions like: should we limit our dependency management to fetching libs? Should we include a module requirement mechanism? Should we encourage asynchronous script loading? Should that be configured or declarative?

Lineman's plugin system takes some of the pressure off needing to make a hasty decision about which packaging horse to back. For starters, we have a terrific Bower plugin and another for Browserify. Read more about Lineman plugins here.

There's simply way too much variety and churn in this space to pick a path and be confident that it'll have been the right decision multiple years from now (in fact, our money is that none of the current approaches will mature into the eventual browser-supported standard in a forward-compatible way). I've written about this topic at length and why I view it as a concern that's not critical enough to concern myself with yet at our blog

About Lineman

The Name

Lineman got its name from finding that the word "grunt" was first used to describe unskilled railroad workers. Grunts that made the cut were promoted to linemen.