Preparing For Release

(This is part of the process for releasing Google-owned code as open source. There are different processes for patches and IARC projects.)

Google believes in open source and wants to contribute back to the community. At the Open Source Programs Office, it's our job to help you do that. We want to approve your request! Before requesting approval to release code, you'll need to do a little prep work for your project.

NOTE: Some groups have slightly different processes for work-related releases. If you're in one of those orgs, you should start by following that process. After satisfying any specific prerequisites, you will end up back at go/releasing to follow the company-wide processes.

• GCP: Start at go/oss-gcp
• DevRel: Expedited approval process.

Name your project

Naming your project (and associated repository) is an important part of releasing. We recommend a descriptive, clear name that's indicative of what the project is. Follow go/naming if you're naming a high-profile project (e.g., the release of the project is being accompanied by a press release or you expect the project to receive significant media attention).

Follow these guidelines specific to open source projects:

• Use a descriptive name that makes it clear what your project does.
• Provide a second-choice name in case the first-choice can't be approved.
• Don't use a Google brand (e.g. Google, Android, YouTube) in the name unless it's a product Google will be promoting.
• Don't use "G-Naming" (e.g. Gmail)
• Don't use third-party brands except when necessary; use them only as descriptors (e.g. "Test Libraries for Java" rather than "Java Test Libraries")
• Don't name the project after anyone else's trademarks, no matter how clever it seems.

For repository names, avoid just concatenating words and instead separating them with hyphens:

DON'T: successortoserverless

DO: successor-to-serverless

License your project

The Apache License is Google's default (it provides us the best legal protections while giving others wide freedom) and you must use it unless there is a specific reason not to, such as:

• the main audience for your release is a community which has standardized around another license

If you think you need a license other than Apache 2, ask emailremoved@ to confirm this for you.

Package your project

Please be very careful when releasing code — read all the comments to ensure they're fit for public viewing and ensure the code has no security implications for Google that haven't been reviewed by the security team. Don't rely solely on the OSPO team's review; we will flag any issues we come across, but our primary focus is on licensing and legal issues. We can't catch every problem.

Third-party components

Google-managed source repositories use top-level /third_party subdirectories to house non-Google-owned files. Making third-party code easy to find helps us and downstream users comply with license requirements.

For any first-party Google open source projects, any code that Google does not own the copyright to or that Google has not received under Google's Contributor License Agreement must be placed in a third_party directory at the root of the project.

For any repositories forked from third-party open source projects, code present in the upstream repository can continue to live in its original location. These projects may also use different layouts or naming conventions, such as calling the subdirectory for third-party code vendor instead of 'third_party'.

Every directory inside third_party (or 'vendor', etc.) needs to have a LICENSE file that includes the full license text and copyright notice for the library.

The repository structure should look like this:

[Root Directory]
|-- Google source code
|-- ....
`-- third_party
    `-- [external library A]
    |   |-- `LICENSE`
    |   `-- ...
    `-- [external library B]
        |-- `LICENSE`
        `-- ...

NOTE: The Unity game engine has restrictions on project folder structure that make it impossible to follow this convention. Specifically, all Unity projects must have all code in an Assets folder at the project root. If a repo contains Unity projects those projects do not need to follow this convention and instead will get manually reviewed.

Third-party components included via package manager

If your project uses a package manager for downloading and using dependencies, please use the appropriate method for defining your dependencies and ensure none of the third-party code is included in your final repo.

Port the build

If your package uses Blaze, you'll need to ensure it works with Bazel (the externally-available, open source version of Blaze) or convert it to another system. To see examples of how other projects have handled this, browse projects on https://github.com/google. If you are maintaining code both internally and externally, Copybara provides workflows and tools that can help you migrate changes from one repository to the other.

Use inclusive language

Google has adopted a policy to use respectful terminology in our internal code and documentation, and the same should be true of our open source projects. We want our projects and communities to be welcoming to everyone, and part of creating an inclusive environment is being thoughtful in the language we use.

All new OSS project should avoid the use of the terms:

• master/slave
• whitelist/blacklist

Instead, use alternatives like those suggested at the links above. Sometimes these terms are unavoidable (for example, to maintain compatibility with an external API) and exceptions will be reviewed on a case-by-case basis. Exceptions will be reviewed as part of the Ariane launch review, and many potential issues can be identified using go/cross.

All new OSS projects must comply with this policy and we are working with existing projects to make changes where possible.

We strongly recommend using tools such as In Solidarity to call out non-inclusive language during reviews.

Google's guide on Writing Inclusive Documentation provides additional advice on avoiding things like ableist and unnecessarily gendered language. Guidance for cleaning up existing code and documentation, as well as more suggested alternative language, can be found at go/respectful-code-existing-systems.

NOTE: Inclusive language will be enforced on all new open source projects starting later in 2020. Cleanup of existing projects will happen over time in coordination with the relevant teams.

Scrub the comments

Once something is checked into version control, it stays visible in the history even if you delete it later.

• Remove names and email addresses of Googlers unless they give explicit permission.
• Remove confidential information, including references to code names, internal paths / filenames, and internal hosts or IP addresses.

TIP: You do not need to scrub bug numbers or Piper CL numbers

Useful commands:

Show google usernames contained in TODO comments:

egrep -r 'TODO\(([^b]|b[^/])[^)]*\)' <path-to-source-directory>

Show references to *.google.com hostnames, @google.com email addresses, google3, and IP addresses:

grep -rP '\.google\.com|@google\.com|(?<!github.com/)google3?/|([0-9]+\.){3}[0-9]+' <path-to-source-directory>

Show Java/C/C++/Go/Objective-C/Objective-C++/Swift/Kotlin comments:

find <path-to-source-dir> -type f | egrep '\.(c|cc|h|cpp|go|java|kt|m|mm|swift)' | while read f; do echo "------------ $f ------------------"; sed -n -e '/\/\*.*\*\// {p; b}' -e '/\/\*/,/\*\//p' -e '/\/\//p' "$f"; done

Show Python/Bash comments:

find <path-to-source-dir> -type f | egrep '\.(py|sh)' | while read f; do echo "------------ $f ------------------"; grep -o "#.*" "$f"; done

Show python docstrings:

#!/usr/bin/env python3
import ast
import os
import sys

class Visitor(ast.NodeVisitor):
  def visit(self, node):
    try:
      docstring = ast.get_docstring(node)
    except TypeError:
      docstring = None
    if docstring:
      print(docstring)
      print('---')
    super().visit(node)

for d, _, filenames in os.walk('.'):
  for name in filenames:
    if not name.endswith('.py'): continue
    p = os.path.join(d, name)
    print(p)
    with open(p) as f:
      Visitor().visit(ast.parse(f.read()))
    print('\n')

Include a README file

TIP: https://github.com/google/new-project contains a repository you can copy with boilerplate README.md, Apache 2 LICENSE, and CONTRIBUTING.md files.

We want people to actually use code released as open source, so include a README file to help people get started. State plainly what your project does. Show examples. Additional documentation is strongly encouraged in a format of your choosing, but a README file is the minimum you should offer. Many hosts will render Markdown syntax if the file is named README.md.

Include LICENSE file and source code headers

Every file containing source code must include copyright and license information. This includes any JS/CSS files that you might be serving out to browsers. (This is to help well-intentioned people avoid accidental copying that doesn't comply with the license.)

BEST PRACTICE: Add license headers to any files that can be added to (i.e. anything that takes the format of a source file and supports file comments). This would include things like YAML files.

NOTE: License headers are generally not required for generated code (i.e. protoc output) unless the tool generating the code is applying its own headers or otherwise asserting that the generated code is a derivative of the tool or otherwise subject to its license.

• For files in the third_party dir, leave the existing headers in place. If you've modified a file, append "Google LLC" to the copyright line.

• For Google-authored source files, paste the below Apache header text in the comments at the top. (Not using the Apache License? We also have other boilerplate available for you to copy.)

• For example code in documentation please include the Apache header below unless the sample code is less than 100 lines, in which case you can use the spdx header described in go/patching

• For files that use prefix-style comments (e.g. # or //) license header comment blocks must not have blank lines in the middle.

  • Good:

    # Paragraph one is on
    # multiple lines.
    #
    # Paragraph two isn't.
    

  • Bad:

    # Paragraph one is on
    # multiple lines.
    
    # Paragraph two isn't.
    

• The addlicense tool automatically recognizes the source files that need a license header and recursively traverses all files to add a license header:

  $GOPATH/bin/addlicense -c "Google LLC" -l apache .
  

• Alternatively, the autogen tool (written by a Googler) helps with this. For example, to license it as "Google LLC" with the Apache License, run it as:

  autogen --no-code --no-tlc -c "Google LLC" -l apache [filename]
  

  or in batch (for Java files, e.g.)

  find . -type f -name \*.java -exec autogen -i --no-code --no-tlc -c "Google LLC" -l apache {} \;
  

• If you use VSCode, you can use Copyright inserter (also written by a Googler) extension to insert the copyright and license header into editing files.

(Planning to actively accept outside patches? See the AUTHORS and CONTRIBUTORS page for info about using "Copyright 2014 The [Project] Authors".)

Apache Header

Copyright 2021 Google LLC

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    https://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

License File

There must also be a file named LICENSE in the top-level directory that contains a copy of the license. For the Apache License, simply copy the official 2.0 license into your LICENSE file verbatim (do not edit the Copyright section in the template, only when you use that template in the header of your source code files.) If you have approval to use a different license, copy the official license for GNU GPL version 2 or GNU LGPL version 2.1 as appropriate, or copy our BSD license.

Include a CONTRIBUTING file

NOTE: This can either be CONTRIBUTING.md or docs/contributing.md in your repository.

The new-project repository provides a basic CONTRIBUTING.md file to get you started.

You must include the Contributor License Agreement (CLA) section unaltered.

Feel free to adjust other parts of the file to suit your project, including instructions for how you would like people to contribute. Do you prefer that they always open an issue before sending large pull requests? Are there specific test suites or linters they should run? This is the place to mention those.

Include a code of conduct (optional)

NOTE: This can either be CODE_OF_CONDUCT.md, docs/CODE_OF_CONDUCT.md, or .github/CODE_OF_CONDUCT.md in your repository.

All Google open source projects are covered by our community guidelines which define the kind of respectful behavior we expect of all participants. However many projects, particularly those with larger communities, may benefit from having an explicit code of conduct included in the project itself.

Projects that would like to include an explicit code of conduct should use the Google modified version of the Contributor Covenant which can be found at CODE_OF_CONDUCT.md. It has been modified to use Google's standard conflict resolution policy as well as broaden the scope to cover any behavior that may negatively impact the project.

To use this code of conduct:

• Select a Project Steward who will be responsible for receiving and addressing reported violations of the code of conduct. They will be the primary point of conduct for the community, but should utilize the emailremoved@ group when needed to help resolve issues.
• Copy CODE_OF_CONDUCT.md to the your project's repository as CODE_OF_CONDUCT.md or docs/code-of-conduct.md. You may also wish to include it in other prominent locations such as a project website. You must modify the Conflict Resolution section to identify the Project Steward and the name of the project. No other modifications should be made.

If you have questions about the code of conduct, or have a good reason for wanting to use a different one, please contact emailremoved@.

Include a security policy (optional)

We provide a security policy template for GitHub repos and other vulnerability reporting paths.

Follow the recommendations in the OSS Vulnerability Guide to create and maintain a coordinated vulnerability disclosure (CVD) process.

If you need additional assistance creating or implementing a security policy or CVD process, fill out the form at go/ospo-strategy-help.

Review your project for issues

Next, run this program against your local copy of your repository:

/path/to/.../cross /path/to/.../location

NOTE: You may need to install BinFS (go/binfs) if this path isn't available to you (for example, if you're using a workstation without a full eng image).

Cross will help you find licensing bits that might not be right, or files with no licenses, or code not copyright Google that should be located in third_party/. It's not foolproof, and you don't need to necessarily take action against all messages (i.e. treat messages as warnings, not errors).

There is no need to paste the output of cross into your launch. We will automatically upload a comment to your Ariane launch with the Cross results once you publish your launch.

Choose your tools and workflows

If you are planning to regularly mirror the source from Google internal repos to public ones (or vice versa) Copybara provides workflows and tools for this.

Stage your code for review

Make your code available so it can be reviewed by the open source releasing and compliance teams. Your project may be eligible to stage directly on GitHub, otherwise you can use either git-on-borg (an internal git repository) or a Cloud Source Repository.

Why move code in Piper to //third_party?

We recommend moving code to //third_party (from elsewhere in Piper) before releasing it. If you ever accept external contributions, this is mandatory.

By making the change up front, it allows you to easily merge external contribution in without having to reorganize the code at that point. It's also an antipattern to have a LICENSE file outside of //third_party, and there are some presubmit checks that may get unhappy.

Even if you're not expecting to accept outside contributions, it's a good idea to do this anyway. What if someone sends a pull request that's too good to refuse?