CrewAI PR Agent - Accelerate Pull Requests with Code PR

CrewAI PR Agent - Accelerate Pull Requests with Code PR
Complete flow of PR Agent

Introduction: Integrating CrewAI to Convert #TODO Comments into Linear Tasks using Pull Requests with PR Coding

I use #TODO comments in commit messages to flag future tasksTracking these tasks manually is inefficient and error-prone.

I am trying to build using CrewAI an PR Agent that pulls these TODOs Requests from Github code & converts into Linear tasks. Also, I added a trigger to run PR agent every time I push a commit using CrewAI.

We also tried doing this with Autogen earlier, but did not connect triggers and I also wanted to measure accuracy on both implementations so will be writing about it later.

TL;DR: Show me the Code

Setup CrewAI PR Coding Agent

!pip install crewai composio_crewai --quiet

Initializing PR Agents using CrewAI

  1. Configure LLM: Use the gpt-4-1106-preview model. Provide the OpenAI environment key through an environment variable or modify the code directly.
  2. Set Up CrewAI PR Agent: Create it with a system prompt and the LLM configuration. Adjust as needed to improve outcomes.

Goal = Take action on Linear via Linear APIs based on Github commits. Linear Project to create issues: Hermes

Backstory= You are an AI Agent with access to Github and Linear and wants to keep the Github Code TODOs and Linear in Sync. Linear Project to create issues: Hermes

from crewai import Agent, Task
from composio_crewai import ComposioToolset, App, Action
from langchain_openai import ChatOpenAI

llm = ChatOpenAI(openai_api_key="sk-uPYkz***", model="gpt-4-0613")

composioCrewAI = ComposioToolset([App.GITHUB, App.LINEAR])

agent = Agent(role='Github-Linear TODO Agent',
  goal="""Take action on Linear via Linear APIs based on Github commits. Linear Project to create issues: Hermes""",
  backstory="""You are an AI Agent with access to Github and Linear and wants to keep the Github Code TODOs and Linear in Sync. Linear Project to create issues: Hermes""",
  verbose=True,
  tools=composioCrewAI,
  llm=llm)

Agent Initialisation Code giving power to agent to interact with Github & Linear

Allowing PR agents to interact with Tools is done using below code

composioCrewAI = ComposioToolset([App.GITHUB, App.LINEAR])

tools=composioCrewAI

Enabling Pull Request Triggers for PR Agent using CrewAI

Pull Request start your CrewAI PR agent when an event happens in a connected app. On every commit, github will send a webhook to run the agent.

  1. Create a Server: Use tools like ngrok to expose the local server/poty for testing.

ngrok http 2000

  1. Set Callback URL: Use composio-cli to set a global callback URL pointing to your exposed server.
composio-cli set global-trigger-callback "<https://try.ngrok.io/asdfjh2>"

Replace your ngrok url or server url to get webhooks.

  1. Configure the Trigger: Use composio-cli commands to enable and configure the github_commit_event trigger.
composio-cli enable-trigger github_commit_event

For our use-case we only need github_commit_event, so after running the above command it will ask for some details to enable the pull request triggers.

> Enabling trigger: github_commit_event...

Owner (Owner of the repository): utkarsh-dixit
Repo (Repository name): speedy

✔ Trigger enabled successfully!
from flask import Flask, request

app = Flask(__name__)

@app.route('/webhook', methods=['POST'])
def webhook():
  task = Task(description=f"""Given the following Github patch: {request.json}, create a Linears issues for the TODOs in the patch and assign them to right people. Please read the patch carefully and create issues for the new TODOs only, avoid removed/old TODOs.""", expected_output="A LINEAR issue created for the commit", agent=agent)
  task.execute()
  return 'Payload received and processed', 200

if __name__ == '__main__':
  app.run(port=2000, debug=True)

Python code to receive the webhook and execute the PR agent

Code PR in Action! 🚀

After setup, commit a TODO requests in your GitHub repository. The CrewAI PR agent should identify the pull requests, parse out TODOs, and create corresponding issues in Linear, assigning them accordingly.

Payload from Composio:
{'trigger_id': 'github_commit_event', 'connection_id': '714ee37a-fb3d-4ef4-a2e8-abde5ce228ca', 'payload': {'id': '307cc46d8c7a3ba56e9c855dce338542b26414a9', 'message': 'Update README.md', 'timestamp': '2024-04-01T13:56:03+05:30', 'author': 'kaavee315', 'url': '<https://github.com/kaavee315/ML_assignment/commit/307cc46d8c7a3ba56e9c855dce338542b26414a9>'}}

> Entering new CrewAgentExecutor chain...
I should fetch the patch file for the given commit to identify any TODOs in the code.

Action: 
github_get_patch_for_commit

Action Input: 
{"owner": "kaavee315", "repo": "ML_assignment", "sha": "307cc46d8c7a3ba56e9c855dce338542b26414a9"}
 

{'execution_details': {'executed': True}, 'response_data': {'patch': 'From 307cc46d8c7a3ba56e9c855dce338542b26414a9 Mon Sep 17 00:00:00 2001\\nFrom: Karan Vaidya <kaavee315@gmail.com>\\nDate: Mon, 1 Apr 2024 13:56:03 +0530\\nSubject: [PATCH] Update README.md\\n\\n---\\n README.md | 1 +\\n 1 file changed, 1 insertion(+)\\n\\ndiff --git a/README.md b/README.md\\nindex 6167076..6c47ce7 100644\\n--- a/README.md\\n+++ b/README.md\\n@@ -1,3 +1,4 @@\\n ML Assignment Readme\\n Change\\n TODO(karan): Fresh set of TODOs\\n+TODO(Utkarsh): Merge the sdk to master\\n'}}

Thought: 
Now that I have the patch, I should parse it to identify any new TODOs that have been added in this commit.

Action: 
None

Action Input: 
None

Action: 
linear_create_linear_issue
------------ ... ---------------------

Final Answer: 
The Linear issue created for the commit is: 

ID: 0d3329f7-1fed-4f01-873c-e44f01cc3e85
Title: Merge the sdk to master
Description: This task is related to the commit 307cc46d8c7a3ba56e9c855dce338542b26414a9 in the ML_assignment repository. The TODO was added by Utkarsh.

> Finished chain.
127.0.0.1 - - [01/Apr/2024 13:57:20] "POST /webhook HTTP/1.1" 200 -

CrewAI PR Agent Output when in Action!

PR Agent Final Code

from flask import Flask, request
from crewai import Agent, Task
from composio_crewai import ComposioToolset, App, Action
from langchain_openai import ChatOpenAI

llm = ChatOpenAI(openai_api_key="sk-uPYkz***", model="gpt-4-0613")

composioCrewAI = ComposioToolset([App.GITHUB, App.LINEAR])

agent = Agent(role='Github-Linear TODO Agent',
  goal="""Take action on Linear via Linear APIs based on Github commits. Linear Project to create issues: Hermes""",
  backstory="""You are an AI Agent with access to Github and Linear and wants to keep the Github Code TODOs and Linear in Sync. Linear Project to create issues: Hermes""",
  verbose=True,
  tools=composioCrewAI,
  llm=llm)

from flask import Flask, request

app = Flask(__name__)

@app.route('/webhook', methods=['POST'])
def webhook():
  task = Task(description=f"""Given the following Github patch: {request.json}, create a Linears issues for the TODOs in the patch and assign them to right people. Please read the patch carefully and create issues for the new TODOs only, avoid removed/old TODOs.""", expected_output="A LINEAR issue created for the commit", agent=agent)
  task.execute()
  return 'Payload received and processed', 200

if __name__ == '__main__':
  app.run(port=2000, debug=True)

Complete Code

Future Plans

I plan to handle small todos directly via agent and let it create PRs for the same. I will publish my results soon in next couple of weeks. Also you can visit our latest blog of building custom AI Agent.

Join our Discord Community and check out what we're building!

Read more