by pledess on 6/2/25, 12:23 AM
For "With your threat model in mind, they should identify opportunities to add new test cases," one common reason is that security engineers are shared across a large company and it may be very expensive for them to learn the different testing frameworks used on many different projects. Also, independent review (without any exposure to developers' conceptions about what should be tested, or why, or how) may be economically justified because outcomes of security bugs are sometimes much worse than outcomes of many categories of ordinary bugs. Other reasons may include that the security engineers want to run a test that can't be expressed in your testing framework without a huge change to the framework, they may want to develop their test cases adaptively such that most of the tests turn out to be useless and the cost of capturing every test under version contol may be very high, they may want to run tests from a commercial testing product for which the license does not allow bulk copying of the tests into a customer's testing framework, or (if they aren't in-house engineers) their business model is that they won't tell you every test that was run unless there's an associated defect finding.