Diagnosing CI test failures

CLI de Copilot ships with the GitHub MCP server, which gives it direct access to your GitHub Actions workflow runs, job logs, and check statuses. Combined with access to your local files, it can fetch CI failure details, correlate them to your code, and propose fixes from your terminal.

Example scenario 1: Tests pass locally but fail in CI

You have a test that passes on your local machine but fails in CI. You can ask CLI de Copilot to investigate this test failure directly.

In this example, the code being tested defines a simple order service (order.py), and there is a corresponding test that checks if an order was created today (test_order_service.py).

Example prompt

My CI is failing on this branch. Can you pull the latest workflow run 
logs, figure out what is failing, and help me fix it? The relevant files 
are @order.py and @test_order_service.py

My CI is failing on this branch. Can you pull the latest workflow run 
logs, figure out what is failing, and help me fix it? The relevant files 
are @order.py and @test_order_service.py

Example response

CLI de Copilot uses the GitHub MCP server to fetch your latest workflow runs on the current branch, identifies the failed job, and retrieves its logs. It finds the following failure:

___ TestOrderService.test_order_created_today ___
>       assert order["created_date"] == date.today()
E       AssertionError: assert datetime.date(2024, 1, 15) == datetime.date(2024, 1, 16)

test_order_service.py:45: AssertionError

After reading both local files, CLI de Copilot notices that the dates are exactly one day apart and identifies this as a timezone boundary issue. The CI runner uses UTC while your local machine uses a different timezone. When the test runs near midnight, date.today() returns different dates in each environment.

CLI de Copilot suggests standardizing how today is computed by injecting a clock dependency or pinning the timezone:

from datetime import timezone, datetime

def get_today():
    return datetime.now(timezone.utc).date()

It applies the fix to both order.py and test_order_service.py, updating all references to use the new get_today() helper.

After the fix, you can verify the changes and run the tests locally:

1. Run /diff to review exactly what CLI de Copilot changed.
2. Run !pytest test_order_service.py to confirm tests pass locally.
3. Push the fix and let CI re-run.

Example scenario 2: Intermittent test failures with race conditions

A test sometimes passes and sometimes fails on the same machine. You want CLI de Copilot to compare logs from passing and failing runs to identify the cause.

In this example, the code under test uses a background job in order_service.py to update an order's status asynchronously, and a test in test_order_service.py asserts that the final status is "processed".

Example prompt for race conditions

This test passes sometimes and fails sometimes. Can you pull the logs 
from the last passing and failing CI runs for test_order_processed 
on this branch, compare them, and figure out what is going wrong? 
Relevant files: @order_service.py @test_order_service.py

This test passes sometimes and fails sometimes. Can you pull the logs 
from the last passing and failing CI runs for test_order_processed 
on this branch, compare them, and figure out what is going wrong? 
Relevant files: @order_service.py @test_order_service.py

Example response for race conditions

CLI de Copilot uses the GitHub MCP server to list recent workflow runs on the current branch, identifies one passing and one failing run, and retrieves the logs for both. It compares them:

Passing run:

[DEBUG] Created order #1234
[DEBUG] Background job started for order #1234
[DEBUG] Background job completed (52ms)
[DEBUG] Checking order status
[DEBUG] Order #1234 status: processed

Failing run:

[DEBUG] Created order #1234
[DEBUG] Background job started for order #1234
[DEBUG] Checking order status
[DEBUG] Order #1234 status: pending

CLI de Copilot spots that in the passing run, the background job completed before the status check, while in the failing run, the status was checked while the job was still running. It identifies this as a race condition because the test does not wait for the background job to finish.

CLI de Copilot suggests adding an explicit wait mechanism before asserting and proposes a fix using a polling helper:

import time

def wait_for_status(order_id, expected, timeout=5):
    start = time.time()
    while time.time() - start < timeout:
        order = get_order(order_id)
        if order.status == expected:
            return order
        time.sleep(0.1)
    raise TimeoutError(
        f"Order {order_id} did not reach '{expected}' within {timeout}s"
    )

Further reading

• • CLI de GitHub Copilot