Compliance Linter

Overview

Compliancelint is an analyzer that runs via Tricorder and shows any findings in Critique; similarly to how golint, error prone, etc. analyses show their findings in Critique. What it does is verify that the information supplied in the third-party metadata files—BUILD, OWNERS, LICENSE, and METADATA—are correct. This allows the programmer to correct mistakes before sending the CL out for review.

Below is some information about the messages the linter emits—what they mean and why they were emitted in the first place.

`BUILD` file warnings

These warnings enforce the third_party BUILD file policies documented at go/thirdparty/documentation#build.

Must have a `BUILD` file

A BUILD file is required for every third party package and must contain a licenses(["<license type>"]) and exports_files(["LICENSE"]) rule.

This error can occur for new projects where a there are no source files to build. In this case we still need a METADATA, OWNERS, LICENSE and BUILD file.

//third_party/example-project/OWNERS
//third_party/example-project/METADATA

//third_party/example-project/BUILD
//third_party/example-project/OWNERS
//third_party/example-project/LICENSE
//third_party/example-project/METADATA

Valid `licenses()` rule

BUILD files must have a licenses() rule which specifies the default license type of the build rules. licenses() should contain the most restrictive license category. The license categories, from most to least restrictive, are:

Compliancelint will identify the licenses in the LICENSE file and check that the category is correct.

For example, if the package contains code covered by two different licenses, one that is "notice" and one that is "restricted", the licenses() rule should have the most restrictive one: "restricted".

licenses(["restricted"])

You can follow theses instructions to identify which license type to specify.

If Compliancelint is unable to identify any licenses in the LICENSE file please add license-escalation to the review line of your CL and ask them for guidance on which license category to use.

No comment for `licenses()` rule

Do not put a comment on the licenses() rule, it can easily become out of sync with the LICENSE file and cause confusion for anyone viewing the BUILD file.

licenses(["notice"]) # MIT

licenses(["notice"])

Must have `exports_files(["LICENSE"])` rule

exports_files() specifies that the LICENSE file is exported to other packages but not otherwise mentioned in the BUILD file.

package(default_visibility = ["//visibility:public"])

licenses(["notice"])

go_library(
    name = "example",
    srcs = ["example.go"],
)

package(default_visibility = ["//visibility:public"])

licenses(["notice"])

exports_files(["LICENSE"])

go_library(
    name = "example",
    srcs = ["example.go"],
)

If you need to export multiple files you can append files to the list, use a glob pattern or define multiple exports_files() rules:

exports_files([ "LICENSE", "ACKNOWLEDGMENTS"])

exports_files(["LICENSE"] + glob(["artefacts/**"]))

Default visibility cannot be public if the license is `by_exception_only`

by_exception_only licenses are normally purchased and licensed only for specific uses. An explicit exception for each Google target build rule is needed before the code can be used by another project. Therefore, the "default" visibility is not allowed for by_exception_only licenses.

package(default_visibility = ["//visibility:public"])

licenses(["by_exception_only"])

exports_files(["LICENSE"])

# *** THIS PACKAGE HAS SPECIAL LICENSING CONDITIONS.  PLEASE
#     CONSULT THE OWNERS AND emailremoved@ BEFORE
#     DEPENDING ON IT IN YOUR PROJECT. ***
package(default_visibility = ["//visibility:private"])

licenses(["by_exception_only"])

exports_files(["LICENSE"])

Anyone wishing to use the package should reach out to emailremoved@ before adding themselves to the package visibility.

`METADATA` file warnings

These warnings enforce the third_party METADATA file policies documented at go/thirdparty/documentation#metadata.

METADATA fields

In a METADATA third_party field, certain fields are required and certain fields are forbidden.

All METADATA files with a third_party field:

name
description

Only METADATA files using type:PACKAGE:

url
version
last_upgrade_date

Only METADATA files in a git repository:

license_type

METADATA URL validation

Compliancelint will examine the content of third_party > url to ensure it's a valid URL, unless the url type is OTHER.

You must specify a `last_upgrade_date` field

The last_upgrade_date field must be set unless the source of truth for the package is Piper and the METADATA file has a url type of PIPER.

Compliancelint will ensure that the last_upgrade_date is added when needed and contains a valid date.

third_party: {
  last_upgrade_date: {
    year: ""
    month: "invalid"
    day: "41"
  }
}

third_party: {
  last_upgrade_date: {
    year: 2006
    month: 1
    day: 2
  }
}

Recent `last_upgrade_date`

The last_upgrade_date field indicates to other Googlers when the package was last updated with an import from the upstream project. This can be a helpful signal to the state of a package.

Please note, last_upgrade_date represents when the upgrade action was performed, not when the software was released.

Compliancelint will check that the last_upgrade_date is within the past month of a CL being created to ensure that the date aligns with this expectation.

Do not use GitHub's archive/master.zip

When defining the URL for a GitHub package do not use the master archive as the URL since is neither a GIT repo URL or an ARCHIVE file, which is intended to be used to point to snapshots of packages.

Instead of using archive/master.zip use the repo url and a type of GIT with the version set to the tag or commit hash.

name: "example-project"
description: "A great example project to use in docs"

third_party {
  url {
    type: ARCHIVE
    value: "https://github.com/example/project/archive/master.zip"
  }
  version: "master"
  last_upgrade_date { year: 2006 month: 1 day: 2 }
}

name: "example-project"
description: "A great example project to use in docs"

third_party {
  url {
    type: GIT
    value: "https://github.com/example/project/"
  }
  version: "929e0e8492d2cfeab4377c5e3d9f1f06b66241d8"
  last_upgrade_date { year: 2006 month: 1 day: 2 }
}

Third-Party package nesting

Third party packages are normally added to the root of //third_party or to the root of a language directory like //third_party/javascript.

There should never be a case where packages are nested (one package under another).

If you are importing a package that includes its dependencies you must separate them out into their own packages.

An example project with nesting could look like this:

//third_party/example_project/METADATA
//third_party/example_project/BUILD
//third_party/example_project/OWNERS
//third_party/example_project/LICENSE
//third_party/example_project/example.c

//third_party/example_project/vendor/other_project/METADATA
//third_party/example_project/vendor/other_project/BUILD
//third_party/example_project/vendor/other_project/OWNERS
//third_party/example_project/vendor/other_project/LICENSE
//third_party/example_project/vendor/other_project/other.c

In this case, the dependency other_project should be moved to its own top level directory:

//third_party/example_project/METADATA
//third_party/example_project/BUILD
//third_party/example_project/OWNERS
//third_party/example_project/LICENSE
//third_party/example_project/example.c

//third_party/other_project/METADATA
//third_party/other_project/BUILD
//third_party/other_project/OWNERS
//third_party/other_project/LICENSE
//third_party/other_project/other.c

Validation of group `METADATA` files

"Group" directories in third-party are directories that contain multiple third-party packages. This is common for directories that group packages by language, for example //third_party/javascript/.

Group directories must have a METADATA file with third_party { type: GROUP }.

Compliancelint will validate these METADATA files and report an issue if the third_party{} find contains information that suggests that the directory is a package itself instead of nesting packages in subdirectories.

For example, a METADATA file with a url, versions and / or last_upgrade_date field would suggest that the directory is a package and not a group.

third_party {
  type: GROUP

  url {
    type: ARCHIVE
    value: "https://github.com/example/repo/release/v1.0.0.tar.gz"
  }

  version: "1.0.0"
  last_upgrade_date { year: 2006 month: 1 day: 2 }
}

If the directory was a package that the fix is to simply remove the type: GROUP attribute, or if the directory is a GROUP then remove these fields will fix the Compliancelint error.

third_party {
  type: GROUP
  group: {
    approval: "b/1234567"
  }
}

See go/thirdparty/metadata#group for more details.

Multiple version validation

When a multiple version exception is granted, the package owner must document the exception in the package METADATA file.

Compliancelint will check the multiple_versions fileld and ensure its content is valid.

Permanent exceptions should not have an expiration field.

third_party{
  multiple_versions{
    type: PERMANENT
    approval: "http://linkremoved/"

    expiration {
      year: 2006
      month: 1
      day: 2
    }
  }
}

third_party{
  multiple_versions{
    type: PERMANENT
    approval: "http://linkremoved/"
  }
}

Do not use `license_type` in Piper

The METADATA protobuf has a license_type field that should only be used for third-party packages on a Gerrit.

Piper must not have this field and the information must be added to the BUILD file in the licenses() rule.

third_party {
  url {
    type: PIPER
    value: "//third_party/example_project"
  }
  license_type: UNENCUMBERED
}

third_party {
  url {
    type: PIPER
    value: "//third_party/example_project"
  }
}

Missing `METADATA` file

When Compliancelint finds a LICENSE file outside of a third-party package it'll raise a finding as this usually indicates that a METADATA file is missing or incorrectly documented.

Example of a project missing a METADATA file:

//third_party/example/project/LICENSE
//third_party/example/project/BUILD
//third_party/example/project/OWNERS
//third_party/example/project/example-code.go

In this case the solution would be to create a METADATA file with a third_party field.

//third_party/example/project/METADATA
//third_party/example/project/LICENSE
//third_party/example/project/BUILD
//third_party/example/project/OWNERS
//third_party/example/project/example-code.go

Another example where this may occur is when a project is nested under a GROUP directory that has a METADATA file with type: GROUP instead of type: PACKAGE.

//third_party/javascript/METADATA
//third_party/javascript/example/project/LICENSE
//third_party/javascript/example/project/BUILD
//third_party/javascript/example/project/OWNERS
//third_party/javascript/example/project/example-code.go

In this case Compliancelint knows that the //third_party/javascript/METADATA file is describing the //third_party/javascript/ directory as a group of packages and infers that //third_party/javascript/example/project/ should have its own METADATA file.

The solution is the same as above, create a METADATA file with a third_party{} field.

//third_party/javascript/METADATA
//third_party/javascript/example/project/METADATA
//third_party/javascript/example/project/LICENSE
//third_party/javascript/example/project/BUILD
//third_party/javascript/example/project/OWNERS
//third_party/javascript/example/project/example-code.go

`OWNERS` file warnings

These warnings enforce the third_party OWNERS file policies documented at go/thirdparty/documentation#owners.

`OWNERS` files in third_party/ should not exist outside of a package

This error triggers when the OWNERS file is placed in an organizational directory, unassociated with a particular package or directories designed to host multiple versions of a project.

OWNERS files are allowed in organizational subdirectories if they only contain per-file directives. This ensures that all new packages go through compliance review.

Do not use `set noparent` in third_party `OWNERS` files

Under no circumstances may an OWNERS file under //third_party include the line set noparent.

In some situations, third-party-removed may need to make changes in //third_party on short notice (often for legal compliance reasons), and barriers such as set noparent files have caused issues in the past.

set noparent

file://third_party/example/OWNERS.team

file://third_party/example/OWNERS.team

File directive should point to relative 'java' `OWNERS` file

OWNERS files under //third_party/java_src are required to have two FTEs listed but must have a file directive pointing to an OWNERS file in the related //third_party/java repository.

For example, if a project resides in //third_party/java_src/example/ and //third_party/java/example/ the //third_party/java_src/example/OWNERS file should only contain:

file://piper/.../OWNERS

`LICENSE` file warnings

These warnings enforce the third_party LICENSE file policies documented at go/thirdparty/documentation#license.

Must have a `LICENSE` file

A LICENSE file is required for every third party package and must contain the license of the project you are importing.

If your project has a commercial license then please copy and paste the contents of the license agreement into the LICENSE file and include a link to the PDF at the top of the file.

//third_party/example-project/OWNERS
//third_party/example-project/METADATA

//third_party/example-project/BUILD
//third_party/example-project/OWNERS
//third_party/example-project/LICENSE
//third_party/example-project/METADATA

In third_party/(kotlin|java)_src, any LICENSE file for a PACKAGE directory must have a corresponding link in third_party/(kotlin|java).

For example:

# For third_party/java_src/myproject/v1/LICENSE
mkdir -p third_party/java/myproject/v1
ln -rs third_party/java_src/myproject/v1/LICENSE third_party/java/myproject/v1/LICENSE

Found a `LICENSE` file in GOOGLE_INTERNAL directory

A "GOOGLE_INTERNAL" directory is a directory inside a third-party package that contains internal code.

A common example is a g3doc directory that is only intended for use by Googlers.

These directories have a METADATA file with third_party { type: GOOGLE_INTERNAL }.

GOOGLE_INTERNAL directories must not have a LICENSE file because they may only contain 100% Google authored code that will never shared externally. If the directory has a LICENSE file that suggests one of two things:

The directory is external code, in which case the type in the METADATA file should change to "PACKAGE".
The LICENSE file was added by mistake and not needed.

Other license files

Third party packages in Google must have a file named LICENSE that contains all license information for a package.

It's not uncommon for external packages to include license information in files namedCOPYING, COPYING.md, LICENSE.txt etc. If that is the case, please rename the file to LICENSE or ensure the text is copied to the packages LICENSE file.

Compliancelint looks for these files and will raise an error if any licenses are missing from the LICENSE file.

For info on how to handle multiple licenses please see go/thirdpartylicenses#multiple

No known licenses

There are scenarios where Compliancelint is unable to detect a license from the LICENSE file, for example due to a commercial license.

When this occurs, please add license-escalation to you CLs review line and ask for the correct licenses condition.

Non-Google copyright holder outside of a package

There should never be a case where a non-Google copyright holder is found in a file outside of a third-party package.

If this occurs then it's a sign that the package has not been documented with a METADATA, OWNERS, BUILD and LICENSE file correctly.

Deleted files warnings

Some but not all of ... were removed

When removing or branching a third-party package you must ensure of the packages metadata files, see the list below, are removed or branched as well.

METADATA
BUILD
LICENSE
OWNERS

FAQ

How can I unblock a CL?

If a CL encounters a problem raised by Compliancelint, you can still request a review by telling critique to run the presubmits and ignore warnings.

First, click "Request Review" or "Modify CL" and add reviewers (add third-party-*removed* if you'd like help fixing Compliancelint).
Then select the "Code review options" tab.
Set the Presubmit option to Run, but ignore warnings.

After this, your CL should go out for review, and you can work with the auto-assigned third-party-*removed* on fixing any Compliancelint issues.

How do I get help?

If you need help to fix an issue there a few options available to you:

👋 CL Reviewer

Add third-party-*removed* to the review list of your CL and someone will be assigned to review your CL and help you with any specific problems you might have.

📧 Email

The email list on emailremoved@ has several Googlers able to help with third party packages at Google.

💬 Chat

Ask any questions you have on go/opensource-chat.

How do I temporarily disable compliancelint?

Before disabling the presubmit, please do one or more of the following:

Use the support channels available to get help on the issue.
Raise a bug at go/compliancelint-bug to fix the problems you are experiencing with Compliancelint.
Request an exception if your package can't follow the policies enforced by Compliancelint.

If you are unable to resolve the issue, you can disable specific checks by using the IGNORE_COMPLIANCELINT tag and getting an LGTM from third-party-removed.

Add the following tag to the end of your CL description.

IGNORE_COMPLIANCELINT=<Check Name>: <Rationale>

You can disable multiple checks like so:

IGNORE_COMPLIANCELINT=<Check Name>: <Rationale>; <Check Name>: <Rationale>

Add third-party-*removed* to the review line of your CL.
Wait for an LGTM from the auto-assigned third-party-removed.

Compliancelint will still lint your CL, but it'll treat findings for these checks as non-actionable once you have an LGTM from the third-party-removed.

TIP: Description tags must be the last thing in the CL description, or Piper may not parse them correctly.

The check names are marked by ❌.

For example, if Compliancelint had the following report:

Compliancelint found 1 failing check(s).
Please review and repair the following issues:

❌ ExampleFailingCheck
    -    This is not a real check, but demonstrates how the report will
         look.
         🗒️ //piper/.../BUILD
            Line(s): 6
         📖 See [go/thirdparty/linter](/documentation/reference/thirdparty/linter/)

❌ AnotherFailingCheck
    -    This is not a real check, but demonstrates how the report will
         look....again.
         🗒️ //piper/.../METADATA
         📖 See [go/thirdparty/linter](/documentation/reference/thirdparty/linter/)

❓Get Help
    📖 See [go/thirdparty/linter](/documentation/reference/thirdparty/linter/) for help on common compliancelint problems
       and how to fix them.
    👋 You can get help from a Googler by emailing `emailremoved@`.

📦 Scopes Examined
    - //piper/.../example

📝 Report Notes
    - 1 packages/directories linted

🛡️ GroupApproval

✅ BuildExists
✅ BuildExportsLicense
✅ CategoryMatchesLicense

In the example above, there are two failing checks, ❌ ExampleFailingCheck and ❌ AnotherFailingCheck. If we wanted to disable both of these, we would change our CL description to:

Example CL Description

IGNORE_COMPLIANCELINT=ExampleFailingCheck: bug ...34; AnotherFailingCheck: bug ...678

How do I get a permanent exception for a check?

There are scenarios where a package is unable to comply with the policies enforced. Rather than rely on IGNORE_COMPLIANCELINT for all of your changes, please file an exception request providing which directories need the exception and why they can't comply with the go/thirdparty policies.

Compliance Linter

Overview

BUILD file warnings

Must have a BUILD file

Valid licenses() rule

No comment for licenses() rule

Must have exports_files(["LICENSE"]) rule

Default visibility cannot be public if the license is by_exception_only

METADATA file warnings