AI Agent for GitHub: Automate Code Review, PR Summaries & Issue Triage

Engineering teams spend 6+ hours per week on code review. An AI agent handles the mechanical parts — style checks, summary generation, security scanning — so human reviewers focus on architecture and business logic.

Why Code Review Needs AI Agents

Code review is essential but painful. Here's the reality:

• Waiting time: Average PR sits 24-48 hours before first review. That's a context switch for the author when feedback arrives.
• Shallow reviews: 60% of code review comments are about style, formatting, or naming — things a linter could catch.
• Review fatigue: After reviewing 400 lines, reviewers catch 50% fewer bugs. Large PRs get rubber-stamped.
• Knowledge silos: Only 1-2 people know each part of the codebase well enough to review it properly.

An AI agent doesn't replace human review. It amplifies it:

6 GitHub Agent Use Cases

Build: PR Review Agent

// github-review-agent.js
import Anthropic from "@anthropic-ai/sdk";
import { Octokit } from "@octokit/rest";

const anthropic = new Anthropic();
const octokit = new Octokit({ auth: process.env.GITHUB_TOKEN });

async function reviewPR(owner, repo, pullNumber) {
  // Get PR details
  const { data: pr } = await octokit.pulls.get({
    owner, repo, pull_number: pullNumber,
  });

  // Get the diff
  const { data: files } = await octokit.pulls.listFiles({
    owner, repo, pull_number: pullNumber, per_page: 100,
  });

  // Build review context
  const changedFiles = files.map(f => ({
    filename: f.filename,
    status: f.status, // added, modified, removed
    additions: f.additions,
    deletions: f.deletions,
    patch: f.patch?.substring(0, 5000), // Limit patch size
  }));

  const totalChanges = files.reduce(
    (sum, f) => sum + f.additions + f.deletions, 0
  );

  // Get repo context (README, contributing guide)
  const repoContext = await getRepoContext(owner, repo);

  // Review with Claude
  const response = await anthropic.messages.create({
    model: "claude-sonnet-4-20250514",
    max_tokens: 4000,
    messages: [{
      role: "user",
      content: `Review this pull request.

PR Title: ${pr.title}
PR Description: ${pr.body || "No description"}
Author: ${pr.user.login}
Base: ${pr.base.ref} ← ${pr.head.ref}
Total changes: +${totalChanges} lines across ${files.length} files

Repository context:
${repoContext}

Changed files:
${changedFiles.map(f =>
  `\n### ${f.filename} (${f.status}, +${f.additions}/-${f.deletions})\n\`\`\`diff\n${f.patch || "Binary file"}\n\`\`\``
).join("\n")}

Provide your review as JSON:
{
  "summary": "2-3 sentence summary of what this PR does",
  "risk_level": "low|medium|high|critical",
  "risk_explanation": "why this risk level",
  "inline_comments": [
    {
      "file": "path/to/file.js",
      "line": 42,
      "side": "RIGHT",
      "severity": "critical|warning|suggestion|nitpick",
      "comment": "what's wrong and how to fix it"
    }
  ],
  "general_feedback": "overall architecture/approach feedback",
  "missing_tests": ["list of scenarios that should be tested"],
  "security_concerns": ["any security issues found"],
  "approved": true/false
}

Rules:
- Focus on bugs, security, and logic errors (high value)
- Don't nitpick style if there's a formatter configured
- Be specific: reference exact lines and suggest fixes
- If the PR is fine, say so — don't invent problems
- Flag breaking changes prominently`
    }],
  });

  const review = JSON.parse(response.content[0].text);

  // Post PR summary comment
  await octokit.issues.createComment({
    owner, repo, issue_number: pullNumber,
    body: formatReviewSummary(review),
  });

  // Post inline comments
  for (const comment of review.inline_comments) {
    try {
      await octokit.pulls.createReviewComment({
        owner, repo, pull_number: pullNumber,
        body: `${severityEmoji(comment.severity)} **${comment.severity.toUpperCase()}**: ${comment.comment}`,
        path: comment.file,
        line: comment.line,
        side: comment.side,
        commit_id: pr.head.sha,
      });
    } catch (e) {
      // Line might not be in the diff — post as general comment
      console.warn(`Could not post inline comment on ${comment.file}:${comment.line}`);
    }
  }

  // Submit the review
  await octokit.pulls.createReview({
    owner, repo, pull_number: pullNumber,
    event: review.approved ? "APPROVE" : "REQUEST_CHANGES",
    body: review.approved
      ? "✅ AI review passed. Human review still recommended."
      : "⚠️ Issues found — see inline comments.",
  });

  return review;
}

function severityEmoji(severity) {
  return { critical: "🔴", warning: "🟡", suggestion: "🔵", nitpick: "⚪" }[severity] || "💬";
}

function formatReviewSummary(review) {
  const riskEmoji = { low: "🟢", medium: "🟡", high: "🟠", critical: "🔴" };
  return `## 🤖 AI Code Review

**Summary:** ${review.summary}

**Risk Level:** ${riskEmoji[review.risk_level]} ${review.risk_level.toUpperCase()} — ${review.risk_explanation}

**Inline Comments:** ${review.inline_comments.length} (${review.inline_comments.filter(c => c.severity === "critical").length} critical)

${review.security_concerns.length > 0
  ? `### 🔒 Security Concerns\n${review.security_concerns.map(c => `- ${c}`).join("\n")}\n`
  : ""}
${review.missing_tests.length > 0
  ? `### 🧪 Missing Tests\n${review.missing_tests.map(t => `- ${t}`).join("\n")}\n`
  : ""}
${review.general_feedback ? `### 💡 General Feedback\n${review.general_feedback}\n` : ""}
---
*This is an automated review. Human review is still required before merge.*`;
}

Build: PR Summary Generator

async function generatePRSummary(owner, repo, pullNumber) {
  const { data: files } = await octokit.pulls.listFiles({
    owner, repo, pull_number: pullNumber, per_page: 100,
  });

  const diff = files.map(f =>
    `${f.filename} (${f.status}, +${f.additions}/-${f.deletions})`
  ).join("\n");

  const response = await anthropic.messages.create({
    model: "claude-sonnet-4-20250514",
    max_tokens: 800,
    messages: [{
      role: "user",
      content: `Generate a PR summary for reviewers. Be concise.

Files changed:
${diff}

Return in this format:
## What Changed
(2-3 bullet points explaining the changes at a high level)

## Why
(1 sentence on the motivation)

## Review Focus
(What reviewers should pay attention to)

## Impact
(What could break, who is affected)`
    }],
  });

  // Update PR description
  const { data: pr } = await octokit.pulls.get({
    owner, repo, pull_number: pullNumber,
  });

  await octokit.pulls.update({
    owner, repo, pull_number: pullNumber,
    body: `${pr.body || ""}\n\n---\n\n🤖 **Auto-generated Summary:**\n\n${response.content[0].text}`,
  });
}

Build: Issue Triage Agent

async function triageIssue(owner, repo, issueNumber) {
  const { data: issue } = await octokit.issues.get({
    owner, repo, issue_number: issueNumber,
  });

  // Get repo labels
  const { data: labels } = await octokit.issues.listLabelsForRepo({
    owner, repo, per_page: 100,
  });

  // Search for related issues
  const { data: related } = await octokit.search.issuesAndPullRequests({
    q: `repo:${owner}/${repo} ${issue.title}`,
    per_page: 5,
  });

  const response = await anthropic.messages.create({
    model: "claude-sonnet-4-20250514",
    max_tokens: 500,
    messages: [{
      role: "user",
      content: `Triage this GitHub issue. Return JSON:
{
  "type": "bug|feature|question|documentation|chore",
  "priority": "critical|high|medium|low",
  "labels": ["from available labels below"],
  "component": "which part of the codebase",
  "duplicate_of": issue_number or null,
  "response": "helpful response to post on the issue",
  "assignee_team": "frontend|backend|devops|design|none"
}

Issue #${issueNumber}: ${issue.title}
Body: ${issue.body?.substring(0, 2000) || "No description"}
Author: ${issue.user.login}

Available labels: ${labels.map(l => l.name).join(", ")}

Potentially related issues:
${related.items.slice(0, 5).map(i =>
  `#${i.number}: ${i.title} (${i.state})`
).join("\n")}

Rules:
- Only use labels that exist in the repo
- If it looks like a duplicate, reference the original issue
- Response should be helpful and welcoming
- For bugs: ask for reproduction steps if missing
- For features: acknowledge and explain the process`
    }],
  });

  const triage = JSON.parse(response.content[0].text);

  // Apply labels
  if (triage.labels.length > 0) {
    await octokit.issues.addLabels({
      owner, repo, issue_number: issueNumber,
      labels: triage.labels,
    });
  }

  // Post response
  await octokit.issues.createComment({
    owner, repo, issue_number: issueNumber,
    body: triage.response + (triage.duplicate_of
      ? `\n\nThis might be related to #${triage.duplicate_of}.`
      : ""),
  });

  return triage;
}

Build: Security Scanner Agent

async function securityScan(owner, repo, pullNumber) {
  const { data: files } = await octokit.pulls.listFiles({
    owner, repo, pull_number: pullNumber, per_page: 100,
  });

  const concerns = [];

  for (const file of files) {
    if (!file.patch) continue;

    const response = await anthropic.messages.create({
      model: "claude-sonnet-4-20250514",
      max_tokens: 500,
      messages: [{
        role: "user",
        content: `Scan this code diff for security issues. Return JSON:
{
  "issues": [
    {
      "severity": "critical|high|medium|low",
      "type": "hardcoded_secret|sql_injection|xss|auth_bypass|
              insecure_dependency|data_exposure|other",
      "line": line_number,
      "description": "what the issue is",
      "fix": "how to fix it"
    }
  ]
}

File: ${file.filename}
Diff:
${file.patch.substring(0, 4000)}

Only report REAL security issues. False positives waste developer time.
Check for: API keys, passwords, tokens in code, SQL string
concatenation, unescaped HTML output, missing auth checks,
insecure crypto, exposed internal URLs.`
      }],
    });

    const result = JSON.parse(response.content[0].text);
    if (result.issues.length > 0) {
      concerns.push(...result.issues.map(i => ({
        ...i, file: file.filename
      })));
    }
  }

  if (concerns.some(c => c.severity === "critical")) {
    // Block the PR
    await octokit.pulls.createReview({
      owner, repo, pull_number: pullNumber,
      event: "REQUEST_CHANGES",
      body: `🔴 **SECURITY: Critical issues found**\n\n` +
        concerns.filter(c => c.severity === "critical").map(c =>
          `- **${c.file}:${c.line}** — ${c.type}: ${c.description}\n  Fix: ${c.fix}`
        ).join("\n\n"),
    });
  }

  return concerns;
}

Build: Auto-Changelog Generator

async function generateChangelog(owner, repo, sinceTag) {
  // Get merged PRs since last release
  const { data: comparison } = await octokit.repos.compareCommits({
    owner, repo,
    base: sinceTag,
    head: "main",
  });

  // Get PRs from merge commits
  const prNumbers = comparison.commits
    .filter(c => c.commit.message.startsWith("Merge pull request"))
    .map(c => parseInt(c.commit.message.match(/#(\d+)/)?.[1]))
    .filter(Boolean);

  const prs = await Promise.all(
    prNumbers.map(n =>
      octokit.pulls.get({ owner, repo, pull_number: n })
        .then(r => r.data)
    )
  );

  const response = await anthropic.messages.create({
    model: "claude-sonnet-4-20250514",
    max_tokens: 1500,
    messages: [{
      role: "user",
      content: `Generate a changelog from these merged PRs.

PRs:
${prs.map(pr =>
  `#${pr.number}: ${pr.title} (by @${pr.user.login})\n${pr.body?.substring(0, 200) || "No description"}`
).join("\n\n")}

Format as:
## [version] — YYYY-MM-DD

### 🚀 Features
- Description (#PR)

### 🐛 Bug Fixes
- Description (#PR)

### 🔧 Improvements
- Description (#PR)

### ⚠️ Breaking Changes
- Description (#PR)

Rules:
- Write for end users, not developers
- Group logically, not by PR order
- Highlight breaking changes prominently
- Credit contributors with @mentions`
    }],
  });

  return response.content[0].text;
}

Tool Comparison: 8 AI Code Review Tools

Tool	Best For	Price	Custom Rules
GitHub Copilot PR Review	Native GitHub integration	$19/user/mo	⚠️ Limited
CodeRabbit	Comprehensive AI code review	Free / $15/user/mo	✅ Good
Sourcery	Python-focused quality checks	Free / $20/mo	⚠️ Some
Codacy	Code quality + security scanning	Free / $15/user/mo	✅ Good
Snyk	Security & dependency scanning	Free / $25/mo	✅ Full
Qodo (CodiumAI)	Test generation from PRs	Free / $19/mo	⚠️ Some
Graphite	Stacked PRs + AI summaries	Free / $20/user/mo	❌ Limited
Custom Agent (this guide)	Full control, any workflow	~$5-20/mo (API)	✅ Full

Our recommendation: Start with CodeRabbit — it's the best out-of-the-box AI code review with good customization. Use GitHub Copilot if you're already paying for it. Build custom when you need specific security rules, integration with internal tools, or custom review logic that no SaaS tool supports.

Complete GitHub Action (Copy & Paste)

# .github/workflows/ai-review.yml
name: AI Code Review

on:
  pull_request:
    types: [opened, synchronize]

permissions:
  contents: read
  pull-requests: write
  issues: write

jobs:
  review:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - uses: actions/setup-node@v4
        with:
          node-version: "20"

      - name: Install dependencies
        run: npm install @anthropic-ai/sdk @octokit/rest

      - name: Run AI Review
        env:
          ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
        run: |
          node -e "
            const { reviewPR } = require('./scripts/ai-review.js');
            reviewPR(
              '${{ github.repository_owner }}',
              '${{ github.event.repository.name }}',
              ${{ github.event.pull_request.number }}
            ).then(r => console.log('Review complete:', r.risk_level));
          "

6 Mistakes That Annoy Developers

What's Next

Start with the PR summary generator — it's the least controversial and immediately useful. Developers love not having to write PR descriptions. Then add the review agent, starting in "advisory only" mode (no blocking). Once the team trusts it, add security scanning as a required check.

The best engineering teams don't resist AI code review. They configure it to match their standards, use it to catch what humans miss, and free up reviewers to focus on what actually matters: "Is this the right approach?"