test(ios): ClientTLS coverage 55.76% -> 89.49%, gate it, fix the three dead CI legs
ClientTLS was the most security-sensitive package in the tree and the least covered, and it was not in the coverage gate at all (the gate's 4-package set predates it). 48 -> 84 tests against the real macOS keychain, serialized with a custom Testing trait after a @globalActor proved insufficient (actors yield at await, so cross-await critical sections got interleaved by other cases' cleanup). CI: the app/ipad/ios17 legs ran a bundle containing LiveServerSmokeTests, which spawns tsx, with no npm ci -- a hard failure, not a skip, on a bare checkout. Adds the missing iPad UI-test leg, and makes a missing iOS 17 runtime fail loudly instead of silently reporting green.
This commit is contained in:
Yaojia Wang
153
.github/workflows/ios.yml
vendored
153
.github/workflows/ios.yml
vendored
@@ -6,8 +6,9 @@
|
||||
# the test binary (a known flaw, fix assigned to T-iOS-16). The corrected
|
||||
# per-package filter lives in ios/IntegrationTests/scripts/coverage-gate.sh
|
||||
# (jq keeps only Packages/<P>/Sources/, excluding *Placeholder*).
|
||||
# 2. app-tests — xcodegen + xcodebuild test (WebTermTests bundle,
|
||||
# iPhone 16 simulator): ViewModels/components of the app glue layer.
|
||||
# 2. app-tests — xcodegen + xcodebuild test (WebTermTests bundle) on
|
||||
# an iPhone AND an iPad simulator: ViewModels/components of the app glue
|
||||
# layer, on both the compact and the regular size class.
|
||||
# 3. integration-tests — Swift Testing against the REAL Node server. The
|
||||
# ServerHarness self-bootstraps `tsx src/server.ts` on an ephemeral
|
||||
# loopback port (127.0.0.1:<free-port> is always in the derived Origin
|
||||
@@ -35,14 +36,15 @@ on:
|
||||
|
||||
jobs:
|
||||
# Layer 1: pure-SwiftPM package tests + the 80% own-sources coverage gate.
|
||||
# The gate covers exactly the 4 gated packages (plan §9); TestSupport runs
|
||||
# tests below without a gate (test doubles are excluded from the gate).
|
||||
# The gate covers the 4 packages of plan §9 PLUS ClientTLS (added by B4 — the
|
||||
# mTLS identity/keychain package, previously the only ungated one at 55.76%).
|
||||
# TestSupport runs tests below without a gate (test doubles are excluded).
|
||||
package-tests:
|
||||
runs-on: macos-15
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
package: [WireProtocol, SessionCore, HostRegistry, APIClient]
|
||||
package: [WireProtocol, SessionCore, HostRegistry, APIClient, ClientTLS]
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- name: Select Xcode 16.3
|
||||
@@ -56,47 +58,66 @@ jobs:
|
||||
- uses: actions/checkout@v4
|
||||
- name: Select Xcode 16.3
|
||||
run: sudo xcode-select -s /Applications/Xcode_16.3.app/Contents/Developer
|
||||
- name: swift test (TestSupport — no coverage gate, plan §9 gates 4 packages)
|
||||
- name: swift test (TestSupport — no coverage gate; it IS the test doubles)
|
||||
run: swift test --package-path ios/Packages/TestSupport
|
||||
|
||||
# Layer 2: app-target unit tests (WebTermTests, hosted by the app).
|
||||
# Layer 2: app-target unit tests (WebTermTests, hosted by the app), on the
|
||||
# compact iPhone path AND the regular iPad path (T-iPad-1: the adaptive
|
||||
# NavigationSplitView layout of T-iPad-2 must be exercised in CI, not only
|
||||
# compact). One matrix instead of two near-identical jobs.
|
||||
#
|
||||
# THE NODE DEPENDENCY (B4 fix): the WebTermTests bundle contains
|
||||
# LiveServerSmokeTests, whose SimServerHarness spawns
|
||||
# `node_modules/.bin/tsx src/server.ts`. On a bare checkout there is no
|
||||
# node_modules, so the harness throws
|
||||
# setup: 找不到 …/node_modules/.bin/tsx — 先在 repo 根目录跑 npm install/npm ci
|
||||
# and the test HARD-FAILS (it is not a skip) — i.e. both legs were red on every
|
||||
# run. Fix: `npm ci` on the canonical iPhone leg, which is the one that owns the
|
||||
# live-server smoke; the iPad leg SKIPS that suite instead of paying a second
|
||||
# node-pty native build. Rationale: the iPad leg exists to exercise the app's
|
||||
# regular-width layout, not to re-verify the client↔server contract, which is
|
||||
# already covered three times over (this leg, integration-tests, ui-test).
|
||||
# `-skip-testing` (rather than a test plan) keeps the change inside this file:
|
||||
# test plans live in ios/project.yml, owned by another task.
|
||||
app-tests:
|
||||
runs-on: macos-15
|
||||
timeout-minutes: 45
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
include:
|
||||
- leg: iphone
|
||||
device: iPhone 16
|
||||
needsNode: true
|
||||
extraArgs: ""
|
||||
- leg: ipad
|
||||
device: iPad Pro 11-inch (M4)
|
||||
needsNode: false
|
||||
extraArgs: "-skip-testing:WebTermTests/LiveServerSmokeTests"
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- name: Select Xcode 16.3
|
||||
run: sudo xcode-select -s /Applications/Xcode_16.3.app/Contents/Developer
|
||||
- uses: actions/setup-node@v4
|
||||
if: matrix.needsNode
|
||||
with:
|
||||
node-version: 22
|
||||
- name: npm ci (LiveServerSmokeTests spawns node_modules/.bin/tsx)
|
||||
if: matrix.needsNode
|
||||
run: npm ci
|
||||
- name: Install XcodeGen
|
||||
run: brew install xcodegen
|
||||
- name: Generate project
|
||||
run: cd ios && xcodegen generate
|
||||
- name: xcodebuild test (WebTermTests, iPhone 16 simulator)
|
||||
- name: xcodebuild test (WebTermTests, ${{ matrix.device }} simulator)
|
||||
# No CODE_SIGNING_ALLOWED=NO: KeychainHostStoreLiveTests exercises the
|
||||
# real data-protection keychain, which returns -34018 for UNSIGNED test
|
||||
# hosts; simulator ad-hoc signing needs no certificates. (W5-fix
|
||||
# handoff finding, verified locally in the ui-test leg runs.)
|
||||
run: |
|
||||
xcodebuild -project ios/WebTerm.xcodeproj -scheme WebTerm \
|
||||
-destination 'platform=iOS Simulator,name=iPhone 16' \
|
||||
test
|
||||
|
||||
# iPad adaptation (T-iPad-1): run the same app suite on an iPad simulator so
|
||||
# the adaptive layout (regular size class / NavigationSplitView, T-iPad-2) is
|
||||
# exercised in CI, not only the compact iPhone path.
|
||||
ipad-tests:
|
||||
runs-on: macos-15
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- name: Select Xcode 16.3
|
||||
run: sudo xcode-select -s /Applications/Xcode_16.3.app/Contents/Developer
|
||||
- name: Install XcodeGen
|
||||
run: brew install xcodegen
|
||||
- name: Generate project
|
||||
run: cd ios && xcodegen generate
|
||||
- name: xcodebuild test (WebTermTests, iPad Pro 11-inch simulator)
|
||||
run: |
|
||||
xcodebuild -project ios/WebTerm.xcodeproj -scheme WebTerm \
|
||||
-destination 'platform=iOS Simulator,name=iPad Pro 11-inch (M4)' \
|
||||
-destination 'platform=iOS Simulator,name=${{ matrix.device }}' \
|
||||
${{ matrix.extraArgs }} \
|
||||
test
|
||||
|
||||
# Layer 3: contract tests against the real Node server (T-iOS-16 test list).
|
||||
@@ -121,9 +142,27 @@ jobs:
|
||||
# runner process reads WEBTERM_SERVER_URL (delivered via xcodebuild's
|
||||
# TEST_RUNNER_ env prefix) and makes its own HTTP assertions against the
|
||||
# server (/live-sessions, /live-sessions/:id/preview, held /hook/permission).
|
||||
#
|
||||
# iPad leg (B4 fix): ProjectsLayoutUITests has an explicit `isPad` branch
|
||||
# (landscape + NavigationSplitView sidebar reveal, T-iPad-4) that NEVER ran —
|
||||
# the only UI leg was iPhone. It now runs on an iPad simulator too. Only THAT
|
||||
# suite: HappyPathUITests has no regular-width branch at all (no sidebar
|
||||
# reveal), so running it on iPad would fail for a missing-feature reason rather
|
||||
# than a regression — making the happy path iPad-adaptive is app-layer work,
|
||||
# tracked separately.
|
||||
ui-test:
|
||||
runs-on: macos-15
|
||||
timeout-minutes: 45
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
include:
|
||||
- leg: iphone
|
||||
device: iPhone 16
|
||||
suites: ""
|
||||
- leg: ipad
|
||||
device: iPad Pro 11-inch (M4)
|
||||
suites: "-only-testing:WebTermUITests/ProjectsLayoutUITests"
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- name: Select Xcode 16.3
|
||||
@@ -159,7 +198,7 @@ jobs:
|
||||
# inputAccessoryView — it only exists while the SOFT keyboard is up.
|
||||
- name: Disable simulator hardware keyboard (KeyBar rides the soft keyboard)
|
||||
run: defaults write com.apple.iphonesimulator ConnectHardwareKeyboard -bool false
|
||||
- name: xcodebuild test (WebTermUITests, iPhone 16 simulator)
|
||||
- name: xcodebuild test (WebTermUITests, ${{ matrix.device }} simulator)
|
||||
# TEST_RUNNER_<VAR> must be an ENV VAR of the xcodebuild process (it
|
||||
# strips the prefix and injects <VAR> into the test-runner process);
|
||||
# passing it as a KEY=VALUE argument makes it a build setting, which
|
||||
@@ -172,7 +211,8 @@ jobs:
|
||||
TEST_RUNNER_WEBTERM_SERVER_URL: http://127.0.0.1:3217
|
||||
run: |
|
||||
xcodebuild -project ios/WebTerm.xcodeproj -scheme WebTermUITests \
|
||||
-destination 'platform=iOS Simulator,name=iPhone 16' test
|
||||
-destination 'platform=iOS Simulator,name=${{ matrix.device }}' \
|
||||
${{ matrix.suites }} test
|
||||
- name: Server log + shutdown
|
||||
if: always()
|
||||
run: |
|
||||
@@ -181,15 +221,21 @@ jobs:
|
||||
|
||||
# Device-matrix floor (plan §9: "iOS 17 最低目标模拟器各一轮"): run the
|
||||
# WebTermTests unit bundle on an iOS 17.x simulator runtime.
|
||||
# CI-ONLY LEG: GitHub macOS runner images ship older Xcode versions whose
|
||||
# iOS 17.x simulator runtime is registered system-wide with CoreSimulator, so
|
||||
# Xcode 16.x can build against it. Local machines are NOT expected to
|
||||
# download the ~8 GB runtime. If the runner image drops the 17.x runtime,
|
||||
# the leg skips WITH A LOUD NOTICE; if the runtime exists, test failures
|
||||
# fail the job (no silent pass).
|
||||
#
|
||||
# CI-ONLY LEG: local machines are NOT expected to hold the ~7 GB iOS 17
|
||||
# runtime, so this leg is never reproducible on a dev box.
|
||||
#
|
||||
# NO SILENT SKIP (B4 fix): this step used to `exit 0` with a ::notice when the
|
||||
# runner image had no iOS 17.x runtime, and the test step was gated on
|
||||
# `if: steps.sim17.outputs.runtime != ''` — so on image drift the whole
|
||||
# deployment-floor round vanished and the job still reported GREEN. A leg that
|
||||
# can silently stop testing is worse than no leg. Now: if the runtime is
|
||||
# missing, it is DOWNLOADED (`xcodebuild -downloadPlatform iOS
|
||||
# -buildVersion`); if the download does not produce a usable runtime, the job
|
||||
# FAILS. The test step is unconditional.
|
||||
ios17-floor-tests:
|
||||
runs-on: macos-15
|
||||
timeout-minutes: 45
|
||||
timeout-minutes: 90 # the runtime download alone can take ~15 min
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- name: Select Xcode 16.3 (build toolchain)
|
||||
@@ -198,23 +244,40 @@ jobs:
|
||||
run: brew install xcodegen
|
||||
- name: Generate project
|
||||
run: cd ios && xcodegen generate
|
||||
- name: Create iOS 17.x simulator (skip-with-notice if runtime absent)
|
||||
- name: Ensure an iOS 17.x simulator runtime (download if the image lacks it)
|
||||
id: sim17
|
||||
env:
|
||||
IOS17_FALLBACK_VERSION: "17.5"
|
||||
run: |
|
||||
runtime="$(xcrun simctl list runtimes | grep -Eo 'com\.apple\.CoreSimulator\.SimRuntime\.iOS-17-[0-9]+' | tail -n 1 || true)"
|
||||
find_runtime() {
|
||||
xcrun simctl list runtimes \
|
||||
| grep -Eo 'com\.apple\.CoreSimulator\.SimRuntime\.iOS-17-[0-9]+' \
|
||||
| tail -n 1
|
||||
}
|
||||
runtime="$(find_runtime || true)"
|
||||
if [ -z "$runtime" ]; then
|
||||
echo "::notice title=iOS 17 floor leg skipped::no iOS 17.x simulator runtime on this runner image (image drift) — the deployment-floor round did NOT run"
|
||||
echo "runtime=" >> "$GITHUB_OUTPUT"
|
||||
exit 0
|
||||
echo "::warning title=iOS 17 runtime absent::downloading iOS ${IOS17_FALLBACK_VERSION} simulator runtime (runner image drift)"
|
||||
sudo xcodebuild -downloadPlatform iOS -buildVersion "$IOS17_FALLBACK_VERSION" || true
|
||||
runtime="$(find_runtime || true)"
|
||||
fi
|
||||
if [ -z "$runtime" ]; then
|
||||
echo "::error title=iOS 17 floor leg cannot run::no iOS 17.x simulator runtime and the download failed — the deployment-floor round did NOT run, so this job fails instead of reporting a false green" >&2
|
||||
xcrun simctl list runtimes >&2
|
||||
exit 1
|
||||
fi
|
||||
udid="$(xcrun simctl create 'iPhone 15 iOS17' 'iPhone 15' "$runtime")"
|
||||
echo "created $udid with $runtime"
|
||||
echo "runtime=$runtime" >> "$GITHUB_OUTPUT"
|
||||
echo "udid=$udid" >> "$GITHUB_OUTPUT"
|
||||
# No CODE_SIGNING_ALLOWED=NO: KeychainHostStoreLiveTests needs a signed
|
||||
# (ad-hoc, certificate-free on simulator) app host — unsigned = -34018.
|
||||
#
|
||||
# -skip-testing LiveServerSmokeTests: same reason as the iPad leg — that
|
||||
# suite spawns node_modules/.bin/tsx and would hard-fail without `npm ci`.
|
||||
# This leg's job is the DEPLOYMENT FLOOR (does the app build and behave on
|
||||
# iOS 17), not the server contract, so it stays Node-free.
|
||||
- name: xcodebuild test (WebTermTests on the iOS 17.x floor)
|
||||
if: steps.sim17.outputs.runtime != ''
|
||||
run: |
|
||||
xcodebuild -project ios/WebTerm.xcodeproj -scheme WebTerm \
|
||||
-destination "platform=iOS Simulator,id=${{ steps.sim17.outputs.udid }}" test
|
||||
-destination "platform=iOS Simulator,id=${{ steps.sim17.outputs.udid }}" \
|
||||
-skip-testing:WebTermTests/LiveServerSmokeTests \
|
||||
test
|
||||
|
||||
Reference in New Issue
Block a user