Automation Intelligence - The AI with ROI! Using AI to automate test automation development. (by John Koenig and a little AI!)

Test Automation Intelligence with AI

Coding with Assisted Generative AI

Why structured AI output reduces time, errors, and rework

In the world of automation, the pace of delivery is often constrained not by vision but by the time it takes to translate that vision into code. SenseTalk scripting from Keysight's Eggplant is powerful and flexible, but like many domain-specific languages, it demands discipline. Standards must be applied consistently, error handling has to be reliable, and every handler needs to be documented in a way that others can reuse. When these steps are done manually, they consume time and introduce inconsistency.

We adopted generative AI to address these challenges. The goal was never to replace engineers but to enable them to work faster and smarter. By guiding AI models with carefully defined standards and guardrails, we are able to produce code that is structured, consistent, and ready for use. The result is a workflow where engineers focus on test logic and automation strategy while the repetitive scaffolding is generated for them.

Manual coding vs AI-assisted coding

Manually writing a handler can take anywhere from twenty minutes for something simple to more than two hours for a complex case. Each step requires recalling standards, formatting comments, adding logging, and ensuring proper control flow. With AI assistance, the same handler can be generated in minutes, often in seconds for simple tasks. Review and integration still take place, but the time savings are significant.

More important than raw speed is the consistency achieved. AI produces handlers that look and behave the same across projects. This reduces rework, shortens review cycles, and improves maintainability over time. Engineers are no longer reinventing the same patterns, and new team members can contribute more quickly because the codebase is uniform.

Online vs offline models

We use both online and offline AI models, each serving a purpose. Online models provide speed, context retention, and advanced reasoning. They anticipate missing details and often generate handlers that are correct on the first pass. Offline models are slower and require more explicit prompting, but they deliver security by ensuring that data never leaves the environment.

Both options outperform manual coding. Online models accelerate development and reduce iteration time. Offline models enforce discipline and consistency even in highly regulated or isolated environments. The choice depends on whether speed or security is the primary requirement, but in either case the outcome is better than human-only coding.

Why offline AI still matters

It is important to note that even when speed is reduced, offline AI still delivers value. Handlers generated offline are parameterized, reusable, and documented. They may take longer to refine, but they provide a consistent framework that reduces long-term maintenance costs. In environments where security and compliance outweigh convenience, this model ensures that teams can still benefit from AI-assisted coding without exposing sensitive data.

Quality improvements

AI assistance improves quality by embedding standards directly into the code. Each handler is generated with documentation, usage examples, parameter descriptions, and notes. Logging is consistent, error handling is explicit, and parameterization prevents the drift that occurs with hardcoded values. The result is a library of reusable building blocks rather than a collection of one-off scripts.

Key takeaways

Generative AI accelerates development, reduces errors, and enforces standards in a way that manual coding cannot match. Even offline AI delivers measurable ROI by producing consistent, reusable, and documented code. Tracking metrics such as time saved, error reduction, and rework avoided is critical to demonstrating value. Revisiting past AI code sessions also allows work to be repurposed for new projects, extending the benefits over time.

The future of automation is not about choosing between humans and AI. It is about structuring the partnership so that humans focus on higher-level strategy and AI provides reliable scaffolding. This approach increases velocity, improves quality, and ensures that automation remains sustainable and scalable across the organization.