IMPORTANT: This document is for code that lives outside of Piper. For code inside Piper see Put the Code in (the Right) //third_party.
TLDR: Put all non-Google code in a "third_party
" directory in your repository.
This policy applies to third-party code in ALL repositories outside of google3,
including all projects on Gerrit, GitHub, etc.
What and why?
All non-Google code goes into the third_party/
directory of your project
Using third-party code saves time and is consistent with our values. To include
any code with non-Google contributions to it in your project, the code must
be placed in the designated third-party folder of your project. By default this
should be a folder called third_party/
in the root directory of your project
repository, but the exact name may vary by project.
Here is a non-exclusive list of code that must go into the designated third_party directory:
- Open source, free software, or commercially-licensed code needs to be in the designated third-party folder.
- Tools or libraries or protocols that are open source, free software, or commercially licensed, need to be in the designated third party folder.
- Derivative works of third-party code
- Library you wrote for your Ph.D. thesis
- Code you wrote at your last job
- Google modifications of third party code
But please do keep in mind that some software is only available under licenses that make it unsuitable for use at Google. Similarly, we generally don’t want to use such unsuitable code for open source projects either.
I just want one file, why can't I keep it with my other code?
Code gets copied, moved, and modified between projects, repositories, branches, etc. Single files placed outside third-party directories cause serious problems for other people trying to comply with open source licenses. People who use your project expect to be able to understand the licensing requirements. If most of your project seems like it's licensed under Apache-2.0, with just one non-compliant file hiding somewhere, the likelihood of your users falling out of compliance is very high.
Putting all code into the third_party/
directory of your project makes it
easier to examine all non-Google code in a project-agnostic way
We respect other people's rights (both in letter and spirit). When we ship a new product or make a new open source release, we need to examine non-Google code to make sure we're complying with the authors' licenses. We also want outside developers to be able to easily comply with our licenses. When tools want to enforce Google-style, they ignore non-Google code. When there's a security warning about non-Google code, we want to scan all non-Google code to find problems and be able to alert people automatically (see go/vomit). All these are easier when non-Google code is in one place, instead of spread throughout the codebase. Hence, you place them in your project's designated third-party directory. And because third-party directories make life easier for the open source team, we have more time to help you.
Special Cases
I need to include a third-party image file in an Android project
In general, all images and assets should appear in the third_party directory of your project with appropriate LICENSE files in place. For Android projects, drawable resources must be located in the res/drawable directory and cannot have accompanying LICENSE files in that directory. Therefore, if you are including third-party image files in your Android project, you must include an ASSETS_LICENSE file in the root directory of your project with the concatenated license text of all the licenses for all third-party images located in the res/drawable directory of your project.
I need to include third-party source code in a Dart package.
In general, all source code should appear in the third_party/ directory of your project with appropriate LICENSE files in place. Dart packages have specific layout requirements. Dart sources must be located in the lib/ directory and non API sources should be located in the lib/src/ directory. Therefore, if you are including third-party Dart source code in your package, it must be located in the lib/src/third_party directory instead of third_party/. For mixed language projects, code that can be in third_party/ should be there.
Get the code
Get the code
When you find code you want to use, get it. This often means downloading from GitHub or elsewhere. Sometimes it can mean negotiating a license with another company and receiving the code another way. If you have any licensing concerns, contact emailremoved@.
Put the code in (the right) third party location
Find the right third party location
Most third-party code lives in a single repository in each Gerrit/other repo. If
you can't find an appropriate third-party location, or one doesn't exist for
your repository yet, create a folder in the root directory of your project
called third_party/
.
IMPORTANT: Android and Chromium have slightly different details.
Repository Name Third party locations Instructions android
external/
,platform/external/
1go/android3p chromium
third_party/
go/chromium3p
Here is a simplified procedure for putting third-party code into the designated third-party directory of your project if one does not already exist:
Create a folder called
third_party/
in the root directory of your repository. For Go repositories, you can use the namevendor/
for better compatibility with the Go build system.Create a sub-directory for each non-google dependency.
In each sub-directory, make sure there is a file called
LICENSE
which contains the appropriate license text for the dependency. If one doesn't exist then create it. More details on this below.Email emailremoved@ any time code is added, deleted, or modified within the designated
third_party/
directory.NOTE: For some repositories (Android, etc), an auto-assigner has been set up. For those, you should use the auto-assignment instructions. If no instructions exist, send code reviews to emailremoved@.
The repository structure will look something like this:
[Root Directory] |-- Google source code |-- .... `-- third_party `-- [external library A] | |-- `LICENSE` | |-- `METADATA` | `-- ... `-- [external library B] |-- `LICENSE` |-- `METADATA` `-- ...
Following community conventions
Some tools and communities have well established tooling/patterns which serve the same purpose of delineating code ownership as outlined in this document, but does not rely on separating this code into a third_party directory.
Yarn
The yarn package manager has built in tooling for
vendoring its usage within a project. This can be done using the yarn policies
command as documented here.
Yarn creates an entry in a project's .yarnrc
file pointing to a single-file
release file committed in the project in the .yarn/releases
folder.
Separate out dependencies
Many open source projects include all of their dependencies. This is prone to
versioning problems, and so we require that you separate out dependencies from
code you import. These dependencies can be replaced by dependencies to other
code in your third_party/
directory. There's a high chance that the
depended-upon libraries are already there.
Why can there only be one version in a third-party location?
The transitive dependencies of most build systems mean that if there are two versions present, eventually someone will try to build a project that depends on both versions. Untangling this after the fact can be very time consuming. Worse, it can stop a project dead that was not involved in submitting either version.
Document the code and its context
By this point, your code should be working with Google integration. When you submit it, there needs to be enough context and background to help us with the requirements for third-party.
First, check in a pristine copy
The first commit should be the version of the code as it was downloaded. This
allows us to track changes, and to revert back to the unedited version (e.g. we
might change a file to work around a Google-specific quirk. If we remove the
quirk, we want to remove the workaround). To make this easy, you may want to
make a copy of this library in another client initially. The CL with the
pristine copy should also contain LICENSE
and METADATA
files (except on
Android, where we require NOTICE
and MODULE_LICENSE_*
). You do not have to
include unused files, and you can move or rename files if necessary, but do not
modify the contents of any files.
LICENSE must contain the full text of the license for the code.
Please note that, to be used at Google, third-party code must have some sort of license.
The license for this code must be in a file named LICENSE
. If it was
distributed like that, you're good. If not, you need to make LICENSE
be a file
containing the full text of the license. If there's another file in the
distribution with the license in it, rename it to LICENSE
(e.g., rename a
LICENSE.txt
or COPYING
file to LICENSE
). If the license is only available
in the comments or at a URL, extract and copy the text of the license into
LICENSE
.
You may optionally document the generation of the LICENSE
file in the
local_modifications
field of the METADATA
file as follows:
local_modifications:
"LICENSE file has been created for compliance purposes. "
"Not included in original distribution."
If there are multiple licenses for the code, put the text of all the licenses
into LICENSE
along with separators and comments as to the applications. For
additional information about how to deal with multiple licenses see the
multiple-licenses section of the go/thirdpartylicenses.
The LICENSE
file is used to automatically generate "About" or "Legal" screens.
Please also wrap the LICENSE
file to 80 characters and replace any non-ASCII
characters with their ASCII equivalents so that such automated processes work
well.
METADATA must list the information about this third-party code.
This allows people to quickly understand what the package is. A METADATA
file
looks like:
name: "Foo Framework"
description:
"The Foo framework allows Bar frobbers to be registered. This avoids "
"memory duplication as in //path/to/alternate/frobber:bar."
third_party {
url {
type: HOMEPAGE
value: "http://linkremoved//"
}
url {
type: GIT
value: "https://github.com/yoyodyne/foo"
}
version: "X.Y.Z"
last_upgrade_date { year: 2014 month: 10 day: 20 }
license_type: NOTICE
local_modifications: "Changed something important."
}
See go/thirdparty/metadata for explanations of the fields and examples.
The METADATA
file is intended to capture all third-party metadata various
teams use to track and analyze third-party packages. All code within
third_party/
is required to have a METADATA
file with a third_party
field
in the top-level directory of the package.
The local_modifications
field is optional and is for informing other engineers
who may use this package about modifications that may introduce behavioral
differences between the checked-in version at Google and the pristine open
source version. If you only make simple changes that do not affect the package's
behavior, such as fixing compilation errors or renaming headers, you can just
say "See git log" or omit the field entirely.
In google3, the license type of a package
is declared in the BUILD file. Unless your project is using
Bazel you don't need a BUILD file, so the license type
should instead be declared in the METADATA
file. If your license type ends up
being by_exception_only
, be sure to follow all the instructions at
go/thirdpartylicenses.
Any other notes important to a Google user of this package must be listed in
METADATA
(see the
proto definition for a full
list of fields).
The non-HOMEPAGE
URL should be to the versioned package you downloaded. Do
not provide an unversioned URL or a URL to the project page.
For projects based on SVN, use the syntax:
url {
type: SVN
value: "http://<project>.<somehost>.com/"
}
version: "<revision>"
For GitHub projects, use the syntax:
url {
type: GIT
value: "https://github.com/<user>/<project>"
}
version: "<hash>"
NOTE: It's occasionally useful to include extra information along with the
package that falls outside of third-party metadata—e.g., instructions on how to
build the package, a list of local modifications made to the package, etc. While
the metadata format is designed to be flexible enough to handle all use cases,
it's not suitable for free-form text. For those teams that need this extra
information, we suggest placing it in a README.google.md
file. It has the
advantage of keeping the data in a human readable format that can also be
rendered as a webpage.
Get reviews
third-party-removed review
The initial submission of third-party code requires review from a member of emailremoved@. Subsequent changes that don't affect licensing don't require review by emailremoved@. You may modify the code as much as you want.
If you're using a Piper or Gerrit repository, you can have a third-party reviewer assigned automatically by setting up the go/gwsq auto-assigner. See the autoassigning page for details on how to set it up. Once set up, the auto-assigner will quickly assign reviewers to changes. Third-party reviewers generally respond within 1-2 business days.
If you have any questions, please contact emailremoved@.
Getting a security review
Third-party code is a hot spot for security vulnerabilities. When adding a new
package that could potentially be used to process third-party / user content or
otherwise carries any security risk, ping the emailremoved@. If it's a part of a
more complex project, file a security review ticket
for the entire thing, so that the security team can review the implications.
This happens asynchronously, so it typically won't delay your launch. If
security team has any concerns, you may be asked to add a security
field to
the METADATA
file. When you update your code, be mindful of security-related
mailing lists for the project and relevant CVE to
update your package.
npm license checking
If you are using npm packages, you are responsible for making sure that all libraries you directly and indirectly use comply with the license policies.
Most npm packages have license metadata, but checking all dependencies by hand is painful.
NOTE: Many npm packages don't have proper license metadata in their package.json file. (Or they may not have it in the version you're using.) You'll need to check the source code of these by hand.
js-green-licenses
It is possible to parse the package.json
file and extract the metadata, but
it's a hairy mess due to the way the file is structured and the potential for
multiple versions to be installed at once.
js-green-licenses will do this for you!
Yarn
Another option is to use the yarn package manager.
TIP: Yarn has other advantages like caching, a sane file format, a deterministic algorithm for installation, offline support, and more!
# In a directory with an existing npm install (i.e. package.json exists).
# 1. Install yarn
npm install yarn
# 2. Set up yarn: It will convert from package.json to yarn.lock
yarn install
# 3. List all packages with their license
yarn licenses list | less
# 4. Summarize licenses
yarn licenses list | grep License | sort -u | cut -d: -f2
Check the output of step #4 for any policy non-compliant licenses, and then you
search the output of step #3 for those licenses to find the affected package(s).
For packages with an UNKNOWN
license, check the source code, as the metadata
is sometimes missing information.
For policy non-compliant licensed packages, if you cannot find an alternative package, you could consider asking the author to re-license under a policy compliant license. (The majority of npm packages use the MIT license, so that is not a bad one to suggest.)
-
platform/external
is allowed only for theplatform/vendor/google_meta
repository, which contains only checked-in METADATA files) ↩