AI may be able to speed up the design of a test system, but faster design also raises a bigger question: how do engineers know when an AI-generated system has earned their trust?
In the final part of this three-part conversation, EEWorld Editorial Director Aimee Kalnoskas talks with Brian Neff, a technical engineer at Liquid Instruments, about verification, traceability, engineering judgment, and where the guardrails around AI-assisted test and measurement should sit.
EEWorld: Welcome to the final part of our series with Brian Neff of Liquid Instruments, Trust but Verify: AI in Test and Measurement. In part one, Brian defined hardware AI and made the case that trust starts with determinism. FPGAs execute the same logic the same way every time, and that is the whole point. In part two, we covered where AI belongs in the test workflow: spec first, test conditions second, code generation last, with human oversight that stays non-negotiable no matter how capable the tools get.
AI can speed up the design side of test and measurement, but that doesn’t make verification any easier. If anything, it makes it more urgent. In this final installment, Brian argues that AI-generated systems need to earn trust the way a new engineer does: with more scrutiny, not less, at least at first.
“The goal isn’t blind trust, and it isn’t perpetual skepticism. It’s earned confidence based on demonstrated performance.”
For anyone joining at part three, Brian is a technical engineer at Liquid Instruments and a former U.S. Air Force commander with extensive experience directing multimillion-dollar research and test programs. He specializes in robotics, autonomous systems, and electromagnetic environments, and holds a PhD in electrical engineering from the Air Force Institute of Technology.
Brian, if AI accelerates the design phase but verification stays manual, doesn’t that just create a bigger backlog at the stage that was already the bottleneck? How do you avoid that?
Brian Neff: That’s a very real concern. In flight test, delays would compound across the program life cycle. By the time we reached developmental and operational testing, we were already behind and under pressure to finish.
If AI lets an engineer generate designs ten times faster and the verification process doesn’t change, you’ve just shifted the bottleneck, or made it worse. But flip that around. AI removes the inertia of getting started and the time spent on mundane, repetitive tasks. If we’re disciplined, we can move that time and expertise into verification. The value won’t be in generating more designs. It will be in generating more validated designs.
AI can also broaden verification itself. It can suggest operating conditions, edge cases, or failure scenarios an engineer might have overlooked. So it’s more than an implementation tool. It’s a resource for strengthening the verification process, as long as we use it responsibly.
EEWorld: We’ve talked about the newer engineer. An unproven engineer gets more scrutiny from a technical director than a veteran does, and for good reason. Should AI-generated systems be held to the same standard? Stricter traceability, more test cases until they’ve earned trust?
Brian Neff: I think that’s the right analogy. A new engineer isn’t distrusted because they’re incapable. Their capability just hasn’t been demonstrated yet. As they show sound engineering judgment over time, the level of oversight changes.
AI should follow the same path. Early on, AI-generated systems should get greater scrutiny, which means they have to be built so they can be scrutinized. As they prove themselves across thousands of successful applications, that effort can shift elsewhere. The goal isn’t blind trust, and it isn’t perpetual skepticism. It’s earned confidence based on demonstrated performance.
EEWorld: Calculators, simulation tools, and high-level languages raised similar fears about engineers losing touch with fundamentals. What’s actually different this time, and which guardrails from those earlier transitions are worth carrying forward?
Brian Neff: When I was an undergraduate in electrical engineering, every student was expected to learn x86 assembly language so they understood how information propagated through the processor. Today, students know assembly exists, but there’s much less focus on it, because the tools that abstract it away have become so good.
History tells us engineers adopt tools that make them more productive. It takes time to build trust in them, but once that trust exists, nobody wants to give up simulation, automated synthesis, or high-level languages.
What’s different with AI is breadth. A calculator or a compiler touches one part of the process. AI can influence the requirements, the architecture, the implementation, the analysis, and the optimization all at once. That breadth is what makes engineering judgment so much more important, along with recognizing the consequences of accepting a poor decision. It also means deciding where AI should not be part of the process, which I’ll come back to.
EEWorld: For a team lead worried that junior engineers will rely too heavily on AI-generated test plans without building their own intuition, what would you tell them to do, practically?
Brian Neff: I’ve been on both sides of this. In a technical review, a junior engineer would get probing questions about why a test condition was in place or why certain data was being collected. The tech director usually wasn’t curious about the answer. The point was to make sure the team had done its homework.
It’s common to dust off a test plan from a similar effort to get started. Nothing wrong with that. The problem comes when you can’t explain why you’re running those test points or collecting that data.
We have the same opportunity with AI, as long as the tool exposes its reasoning. If it generates a detailed specification and test conditions and lets you interact throughout the process, ask the same questions. Why are we running this test condition? What requirement does it verify? What assumptions were made? What happens if we remove it? That’s how you keep building engineering intuition, and it can happen at every step instead of only in a formidable technical or safety review.
EEWorld: As these tools take on more autonomous roles, moving from generating designs to acting on decisions, what would have to be true before you would trust a system like that?
Brian Neff: It depends on the criticality of the subsystem and the nature of the instrument. I don’t think there’s an ironclad rule for where the guardrail sits.
Here’s an analogy from flight test. We spend months or even years designing aircraft, analyzing failure modes, and debating the best response to every conceivable emergency. There may be several ways to arrive at the right procedure. But once the aircraft is flying and an engine fails or a fire warning appears, nobody wants checklist improvisation in real time. That’s why we have boldface emergency procedures.
“The goal isn’t blind trust, and it isn’t perpetual skepticism. It’s earned confidence based on demonstrated performance.”
Apply that to test and measurement. Our system lets AI help with the specification, the validation, and the generation of the instrument deployed on the FPGA. What I don’t want, as someone who relies on my equipment for deterministic results, is AI continuing to interact with signals and stimuli in a way that could introduce even slight variability in measurement levels or timing. I want the same response, the same way, every time.
So that’s our guardrail: once the instrument is deployed, there’s no more autonomy. Going forward, engineers will have to look at each program individually and decide where that guardrail belongs for the systems they build.
EEWorld: For engineers who are still on the fence, and there’s a lot of healthy skepticism out there, what’s a low-stakes way to test these ideas themselves and form their own opinions rather than taking anyone’s word for it?
Brian Neff: I was in that exact position within the last year, when I learned the company was going to pursue this capability. I was a skeptic, because I’d seen AI used badly. The skepticism is warranted. Let it benefit you, but don’t let it become a barrier to finding the value.
Start with a problem you already know how to solve. We had a customer who needed to buffer incoming data before it reached the triggering algorithms further down the toolchain. I had already built that logic myself. When our generative instrumentation tool came online, I gave it the same request: build a multi-channel data buffer that can hold data up to a set amount of time. That was all the detail I gave it at first.
It started asking questions I hadn’t considered, and the result went from basic functionality to a much more refined instrument. It also found ways to test the instrument that I hadn’t thought of, which made it reliable across a wider range of conditions than my original version.
So don’t start on a critical project where you don’t know what good looks like yet. Test the waters. Then evaluate the result the way you’d evaluate a new engineer’s work. Where did it help? Where did it struggle? What assumptions did it make? What would you change before putting it into practice? If the answers meet your requirements, skepticism can turn into acceptance, and eventually trust.
Don’t blindly accept the hype, and don’t dismiss it outright either. Put these tools through the same engineering process you’d apply to any new technology: measure them, challenge them, validate them.
EEWorld: Starting with a problem you’ve already solved is advice that applies to any AI tool. Brian, any final words for our audience?
Brian Neff: AI allows each engineer to do much more, and just as much opportunity to make mistakes without discipline. Don’t lose sight of where engineering judgment and intuition still matter most.
