Testing Rules

Every TSLint rule has a corresponding directory inside test/rules/ which contains one or more test cases for the rule.

Each test case contains:

  • A tslint.json file which specifies the configuration for TSLint to use
  • .ts.lint test files which contain TypeScript code and a special markup which indicate where lint failures should be found

The test system lints the .ts.lint files with the settings specified in tslint.json and makes sure that failures generated by the TSLint match the failures marked in the .ts.lint files.

Example

Testing a New Rule

Say we’re contributing a new rule to TSLint that bans the use of animal variable names and we now need to test it. First we create our configuration file which enables only the rule to be tested:

test/rules/no-animal-variable-names/default/tslint.json:

{
  "rules": {
    "no-animal-variable-names": true
  }
}

In this case, we placed our configuration inside no-animal-variable-names/default to indicate that we’re testing the default configuration for our rule.

Now let’s make the actual test file:

test/rules/no-animal-variable-names/default/test.ts.lint:

const octopus = 5;
      ~~~~~~~      [Variables named after animals are not allowed!]

let giraffe: number, tiger: number;
    ~~~~~~~                 [Variables named after animals are not allowed!]
                     ~~~~~  [Variables named after animals are not allowed!]

const tree = 5;
const skyscraper = 100;

In the above file, ~ characters “underline” where our rule should generate a lint failure and the message that should be produced appears in brackets afterwards.

If multiple lint failures occur on the same line of TypeScript code, the markup for them is placed on consecutive lines, as shown in the above example with the line let giraffe: number, tiger: number;

Notice how we also include lines of code that shouldn’t generate lint failures. This is important to ensure that the rule isn’t creating false-positives.

We can now run yarn compile:test && yarn test:rules to make sure that our rule produces the output we expect.

Testing a New Rule Option

Let’s look at one more example. Say we’ve added an also-no-plants option to our rule above that disallows variable names that are plants. We should add a test for this new option:

test/rules/no-animal-variable-names/also-no-plants/tslint.json:

{
  "rules": {
    "no-animal-variable-names": [true, "also-no-plants"]
  }
}

test/rules/no-animal-variable-names/also-no-plants/test.ts.lint:

const octopus = 5;
      ~~~~~~~     [no-animal]

let giraffe: number, tiger: number;
    ~~~~~~~                 [no-animal]
                     ~~~~~  [no-animal]

const tree = 5;
      ~~~~      [no-plant]
const skyscraper = 100;

[no-animal]: Variables named after animals are not allowed!
[no-plant]: Variables named after plants are not allowed!

We’ve now used a special message shorthand syntax so we don’t have to type out the same failure message over and over. Instead of writing out the full lint failure message after each occurance of it, we instead just specify a shortcut name. (Shortcut names can only consist of letters, numbers, underscores, and hyphens.) Then, at the bottom of our test file, we specify what full message each shortcut should expand to.

Again, we can run yarn compile:test && yarn test:rules to make sure our rule is producing the output we expect. If it isn’t we’ll see the difference between the output from the rule and the output we marked.

You can also use placeholders to format messages. That’s useful if the error message contains non-static parts, e.g. variable names. But let’s stick to the above example for now.

const octopus = 5;
      ~~~~~~~     [no-animal]

let giraffe: number, tiger: number;
    ~~~~~~~                 [no-animal]
                     ~~~~~  [no-animal]

const tree = 5;
      ~~~~      [error % ("plants")]
const skyscraper = 100;

[error]: Variables named after %s are not allowed!
[no-animal]: error % ('animals')

We created a message template called error which has one placeholder %s. For a complete list of supported placeholders, please refer to the documentation of node’s util.format(). To use the template for formatting, you need to use a special syntax: template_name % ('substitution1' [, "substitution2" [, ...]]). Substitutions are passed as comma separated list of javascript string literals. The strings need to be wrapped in single or double quotes. Escaping characters works like you would expect in javascript.

You may have noticed that the template is used for formatting in two different places. Use it in inline errors to pass another substitution every time. If you use formatting in another message shorthand (like we did for [no-animal]), you need to make sure the template is defined before its use. That means swapping the lines of [error] and [no-animal] will not work. There are no restrictions for the use of [no-animal] in inline errors, though.

Now let’s pretend the rule changed its error message to include the variable name at the end. The following example shows how to substitute multiple placeholders.

const octopus = 5;
      ~~~~~~~     [no-animal % ('octopus')]

let giraffe: number, tiger: number;
    ~~~~~~~                 [no-animal % ('giraffe')]
                     ~~~~~  [no-animal % ('tiger')]

const tree = 5;
      ~~~~      [error % ("plants", "tree")]
const skyscraper = 100;

[error]: Variables named after %s are not allowed: '%s'
[no-animal]: error % ('animals')
Typescript version requirement

Sometimes a rule requires a minimum version of the typescript compiler or your test contains syntax that older versions of the typescript parser cannot handle. When testing with multiple versions of typescript - like tslint does in CI - those tests will fail.

To avoid failing tests, each test file can specify a version requirement for typescript in the first line. If you don’t specify one, the test will always be executed.

Example:

[typescript]: >= 2.1.0

The syntax should look familiar, because it is basically the shorthand syntax from the chapter above. It needs to start with [typescript]:. The following part can be any version range. The prerelease suffix will be removed before matching to allow testing with nightly builds.

Tips & Tricks

  • You can use this system to test rules outside of the TSLint build! Use the tslint --test path/to/dir command to test your own custom rules. You can specify one or more paths to tslint.json files or directories containing a tslint.json. Glob patterns are also supported. Besides the tslint.json each directory you pass should contain *.ts.lint files. You can try this out on the TSLint rule test cases, for example, tslint --test path/to/tslint-code/test/rules/quotemark/single. If you want to test all of your rules at once, you can use it like this: tslint --test rules/test/**/tslint.json

  • To test rules that need type information, you can simply add a tsconfig.json with the desired configuration next to tslint.json.

  • Lint failures sometimes span over multiple lines. To handle this case, don’t specify a message until the end of the error. For example:

for (let key in obj) {
~~~~~~~~~~~~~~~~~~~~~~
  console.log(key);
~~~~~~~~~~~~~~~~~~~
}
~ [for-in loops are not allowed]
  • If for some reason your lint rule generates a failure that has zero width, you can use the ~nil mark to indicate this.

Develop