Implementing Auto-Retry for Flaky Tests

🔄Scenario Overview

Implementing Auto-Retry for Flaky Tests

Key Takeaways & Cheat Sheet

✓Identify causes of flaky tests (network blips, server delay, timing lags)
✓Implement TestNG IRetryAnalyzer interface to govern repeat executions
✓Implement IAnnotationTransformer to apply retry rules globally to all tests
✓Avoid setting massive retry limits that inflate test execution times

Short Direct Answer

To prevent false alarms in CI/CD pipelines caused by temporary network latency or slow API responses, you can implement an auto-retry mechanism. For TestNG, create a class implementing `IRetryAnalyzer` to rerun failed tests a defined number of times (typically 1-2) before marking them as failed.

⚠️ Senior Warning (Red Flag)

Never set high retry limits (e.g. retrying a test 5 times). High retry limits mask real performance bugs and significantly slow down your CI/CD execution pipeline.

💡 STAR Deep Dive Explanation & Pro Tip

Apply this analyzer globally by writing a custom TestNG annotation transformer. This guarantees retry behaviors are active across the entire test suite automatically.

SeleniumAutomation.java

Selenium 4 + Java

import org.testng.IRetryAnalyzer;
import org.testng.ITestResult;

// ✅ Custom Retry Analyzer implementation
public class RetryAnalyzer implements IRetryAnalyzer {
    private int retryCount = 0;
    private static final int MAX_RETRY_LIMIT = 2; // Retry failed tests up to 2 times

    @Override
    public boolean retry(ITestResult result) {
        if (retryCount < MAX_RETRY_LIMIT) {
            retryCount++;
            System.out.println("Retrying failed test: " + result.getName() + " | Attempt: " + retryCount);
            return true; // Rerun the test
        }
        return false; // Stop retrying and mark test as failed
    }
}