mirror of
https://github.com/agent-substrate/substrate.git
synced 2026-11-10 10:13:50 +08:00
Tim Hockin
a8d83ee1b5
This mostly eliminates the need to hand-write validation code. This PR is a long series of commits which add DV for most of Actor and all of Atespace. Here is a map to the commits: * The first few take a dep on a new Kubernetes tag, import the code into third_party, and apply a single patch. Because we have different Go modules for tools, I had to do it twice. When that patch lands, we can revert these commits, but that won't be until the 1.38 cycle in a few months. * The next commits slowly add DV support, so a human can review each of them in a reasonable amount of time. The emphasis is on great test cases. * I made a bad choice early on as to where to generate the code into, so I moved it. Rebasing on that was exceedingly hard, so I left it as a move. * This required changing update/go-generate -> update/codegen -- we need to get the ordering of tools right, which `go generate` does not guarantee. * Then I added a "middle" layer called "ServiceImpl" between the RPC and storage layers. This allows things like workflow to call the same business logic as the RPC layer, including validation. Lots of test fixes. * Then I finished the Create() and Update() paths for Actor. Those represent the "right" (or closest to) way to implement resources, and tests for validation. I strongly encourage reviewers to read it commit-by-commit. Rebasing this is VERY tedious, so the sooner it lands or dies completely, the better. Then we can start converting the rest. `./hack/run-tool.sh validation-gen --docs` will produce some docs on the tool and the available tags. @laoj2 @juli4n @EItanya @HavenXia @lalitc375 @yongruilin @jpbetz FYI
112 lines
3.8 KiB
Bash
Executable File
112 lines
3.8 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
|
|
# Copyright 2026 Google LLC
|
|
#
|
|
# Licensed under the Apache License, Version 2.0 (the "License");
|
|
# you may not use this file except in compliance with the License.
|
|
# You may obtain a copy of the License at
|
|
#
|
|
# http://www.apache.org/licenses/LICENSE-2.0
|
|
#
|
|
# Unless required by applicable law or agreed to in writing, software
|
|
# distributed under the License is distributed on an "AS IS" BASIS,
|
|
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
|
# See the License for the specific language governing permissions and
|
|
# limitations under the License.
|
|
|
|
# Verifies that no Python dependency in any of our requirements.txt files
|
|
# (and their transitive closure) carries a disallowed license. Modeled on
|
|
# hack/verify/licenses.sh (Go side), but Python deps live in per-project
|
|
# venvs that may be ephemeral, so this script creates them on demand.
|
|
#
|
|
# CNCF allowed third-party licenses:
|
|
# https://github.com/cncf/foundation/blob/main/allowed-third-party-license-policy.md
|
|
|
|
set -o errexit -o nounset -o pipefail
|
|
|
|
ROOT="$(git rev-parse --show-toplevel)"
|
|
cd "${ROOT}"
|
|
|
|
# Python projects with a requirements.txt to check. Each entry is a directory
|
|
# that contains requirements.txt; its venv lives at <dir>/venv. Discovered
|
|
# dynamically from tracked/untracked (but not ignored) requirements.txt files,
|
|
# excluding vendored trees and the _LICENSES/ directory.
|
|
PROJECTS=()
|
|
while IFS= read -r line || [[ -n "${line}" ]]; do
|
|
[[ -n "${line}" ]] && PROJECTS+=("${line}")
|
|
done < <(
|
|
git ls-files \
|
|
-cmo \
|
|
--exclude-standard \
|
|
-- \
|
|
':(glob)**/requirements.txt' \
|
|
':!:vendor/*' \
|
|
':!:**/vendor/*' \
|
|
':!:_LICENSES/*' \
|
|
| xargs -r -n1 dirname \
|
|
| sort -u
|
|
)
|
|
|
|
# Fail if any installed package's license string contains one of these
|
|
# semicolon-separated entries. pip-licenses substring-matches, so single
|
|
# substrings like "AGPL" / "GPL" / "SSPL" catch every common spelling.
|
|
# Add entries as needed; prefer this denylist over an allowlist because
|
|
# pip-licenses license strings vary widely (classifier vs. License field
|
|
# vs. PEP-639 expression).
|
|
DISALLOWED="AGPL;GPLv2;GPLv3;GPL-2.0;GPL-3.0;GNU General Public License;GNU Affero General Public License;Server Side Public License;SSPL;Commons Clause"
|
|
|
|
check_project() {
|
|
local dir="$1"
|
|
local venv="${dir}/venv"
|
|
echo "==> ${dir}"
|
|
# Run inside a subshell so `activate` doesn't leak VIRTUAL_ENV/PATH back
|
|
# to the caller; the subshell inherits errexit/nounset/pipefail.
|
|
(
|
|
if [[ ! -d "${venv}" ]]; then
|
|
echo " Creating venv at ${venv}..."
|
|
python3 -m venv "${venv}" || {
|
|
echo "ERROR: failed to create venv at ${venv}" >&2
|
|
exit 1
|
|
}
|
|
echo " Activating ${venv}..."
|
|
source "${venv}/bin/activate"
|
|
echo " Upgrading pip in ${venv}..."
|
|
pip install --quiet --upgrade pip || {
|
|
echo "ERROR: failed to upgrade pip in ${venv}" >&2
|
|
exit 1
|
|
}
|
|
echo " Installing ${dir}/requirements.txt into ${venv}..."
|
|
pip install --quiet -r "${dir}/requirements.txt" || {
|
|
echo "ERROR: failed to install ${dir}/requirements.txt into ${venv}" >&2
|
|
exit 1
|
|
}
|
|
else
|
|
echo " Activating ${venv}..."
|
|
source "${venv}/bin/activate"
|
|
fi
|
|
if ! command -v pip-licenses >/dev/null 2>&1; then
|
|
echo " Installing pip-licenses into ${venv}..."
|
|
pip install --quiet pip-licenses || {
|
|
echo "ERROR: failed to install pip-licenses into ${venv}" >&2
|
|
exit 1
|
|
}
|
|
fi
|
|
echo " Checking licenses in ${venv}..."
|
|
pip-licenses --fail-on="${DISALLOWED}"
|
|
)
|
|
}
|
|
|
|
fail=0
|
|
for proj in ${PROJECTS[@]+"${PROJECTS[@]}"}; do
|
|
if ! check_project "${proj}"; then
|
|
echo "ERROR: ${proj} contains disallowed Python license(s)" >&2
|
|
fail=1
|
|
fi
|
|
done
|
|
|
|
if [[ "${fail}" -ne 0 ]]; then
|
|
exit 1
|
|
fi
|
|
|
|
echo "All Python licenses pass."
|