Sanitizers (TZ 3.5 / 5.2)
OXL_SANITIZE builds the runtime (liboxlrt), the compiler (oxlc, oxl) and every test binary with
AddressSanitizer + UndefinedBehaviorSanitizer, and makes oxl build link the sanitizer runtime into every compiled
OxL program (the program itself is not instrumented, but the instrumented liboxlrt.a inside it is).
cmake -S . -B build-asan -G Ninja -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ \
-DOXL_SANITIZE=address,undefined # or -DOXL_SANITIZE=ON (same thing)
cmake --build build-asan -j2 # ~25 min on 2 cores (instrumented LLVM-heavy compiler)
export ASAN_OPTIONS=detect_leaks=0 UBSAN_OPTIONS=print_stacktrace=1
ctest --test-dir build-asan -j2 --timeout 1800 --output-on-failure --repeat until-pass:2 -E lsp_latency
python3 scripts/run_conformance.py --oxl "$PWD/build-asan/tools/oxl/oxl" --stage 3 -j 2 --quiet \
--timeout 300 --run-timeout 60 --ratchet conformance/baseline-stage3.jsonHow it works:
CMakeLists.txt:OXL_SANITIZEis a string (ON, or a list such asaddress,undefined;threadis passed through but not part of the nightly). It adds-fsanitize=<list> -fno-omit-frame-pointerto compile and link options, and-fno-sanitize-recover=undefined, so any undefined behaviour aborts the process and fails the test instead of scrolling by as a warning.compiler/CMakeLists.txtdefinesOXL_SANITIZE_LINK_FLAG;compiler/src/driver/Build.cppappends it to the link command of compiled programs (otherwise the link fails with undefined__asan_*/__ubsan_*symbols).- Pass an absolute
--oxlpath torun_conformance.py:oxl testkinds change the working directory.
Notes on running under sanitizers:
- Time: unit tests take ~1.7 h of CPU (test_sema 15 min, test_codegen 16 min); the stage-3 conformance run 30 min at -j2.
lsp_latencymeasures wall-clock latency against fixed limits and always fails under ASan; it is excluded.detect_leaks=0for the unit tests: LeakSanitizer works, but many existing unit tests intentionally do not free what they build (hand-built descriptors withoutdrop, etc.). The fuzz harnesses run with leak detection on, which is where leaks in the decoders are checked. Triage of test-only leaks is a known gap.Sched.ManySuspendedTasksStayCheapskips its RSS bound under ASan (shadow memory and redzones).shop_oxl_test(Orders.recent ...) is timing dependent:checkoutclears the cart in the background (~>), so the nextaddToCartof the same user can be overtaken by theclear. It passes ~7 of 8 runs under ASan load and almost always without; not a sanitizer finding.
Bugs found and fixed#
| Where | Finding | Fix |
|---|---|---|
runtime/src/grpc/client.cpp ClientCall::resolveFromStream | heap-use-after-free (ASan, Grpc.BadServers): findHeader(stream_->headers(), ...) returned a pointer into a temporary copy of the headers | keep the copy in a local |
runtime/src/grpc/wire.cpp parseTimeout | signed overflow (UBSan): peer-controlled grpc-timeout: 99999999H = 3.6e20 ns | saturate to INT64_MAX |
runtime/src/ser_json.cpp parseRfc3339 | signed overflow (UBSan, fuzz_ser): timestamps beyond year 2262 / before 1677 in JSON | range check, decoded as DecodeError.Invalid |
runtime/src/http2/h2.cpp Stream::endStream | not idempotent when the stream closed cleanly between two calls (-EPIPE instead of 0); made H2.ApiMisuse flaky under load | no-op on a cleanly closed stream |
tests: config_test.cpp SecretStr.NewRevealDrop | read of a freed string (ASan) | check before the release |
tests: h2_test.cpp H2.WriteDeadline | handler task read a stack variable after the test function returned (stack-use-after-scope) | state lives in a shared_ptr |
tests: grpc_abi_test.cpp UnaryAndMeta | asserted active_calls == 0 immediately after the client saw the status (race) | wait for the server side to finish |
Regression tests: tests/unit/runtime/fuzz_regress_test.cpp, the saturation cases in grpc_test.cpp
(GrpcWire.Timeout); the use-after-free is covered by Grpc.BadServers under OXL_SANITIZE.
Results#
ctestunder ASan+UBSan: all tests pass except the two exclusions above (see the final report for counts).- Conformance stage <= 3 under ASan+UBSan: 1083 baseline tests pass, no ratchet regression (stage 2 is a subset).
- CI:
.github/workflows/nightly.ymljobasan-ubsanruns exactly the commands above.
Source: docs/sanitizers.md