Generative AI capabilities in Power Apps Test Engine (preview)

Power Apps Test Engine offers comprehensive generative AI capabilities that span the entire testing lifecycle. This page provides an overview of how generative AI can enhance your testing experience, from test creation to execution and validation.

Test Engine's generative AI features address three key areas of the testing process:

Generative AI-assisted test authoring

Creating comprehensive test plans can be time-consuming, especially for complex applications. Test Engine supports generative AI-assisted authoring through:

• GitHub Copilot integration: Generate test templates, test steps, and assertions based on your application code
• Natural language test creation: Describe test scenarios in plain English and translate them into executable tests
• Sample-based test generation: Reference existing samples to create contextually relevant tests

This approach helps test authors focus on business logic and validation rules rather than test syntax and boilerplate code.

Model Context Protocol server implementation

Power Apps Test Engine includes a Model Context Protocol (MCP) server implementation that provides deterministic analysis of your applications and generates test recommendations.

The MCP server:

• Analyzes application structure to identify testable components
• Generates test patterns based on control types and relationships
• Provides contextual code recommendations
• Integrates with MCP clients like Visual Studio and GitHub Copilot
• Uses Plan Designer to organize and prioritize testing efforts
• Incorporates solution definition elements and data schemas for comprehensive testing
• Uses metadata from your solution to generate contextually relevant tests

When you combine deterministic analysis with generative AI capabilities, this approach offers more you a more reliable and accurate test generation compared to pure generative approaches alone.

Testing nondeterministic AI capabilities

When testing applications that use AI capabilities like AI Builder components or Generative Pretrained Transformer (GPT) models, special consideration is needed for handling nondeterministic outputs.

Test Engine provides:

• The Preview.AIExecutePrompt function: Execute AI prompts with controlled inputs and validate outputs
• Tolerance-based validation: Verify that AI outputs meet expectations within acceptable thresholds
• Structured response validation: Parse and validate complex AI-generated content
• Plan-based validation: Use Plan Designer definitions to validate AI outputs against expected criteria

These capabilities ensure you can create reliable, repeatable tests even when working with inherently variable AI systems.

Choosing the right generative AI approach

For optimal results, consider these guidelines:

Related articles

AI-assisted test authoring with GitHub Copilot
Using the Model Context Protocol server with Test Engine
Testing nondeterministic AI components
Browse the Test Engine samples catalog
Try the Test Engine power-fx-functions
Use Plan Designer