idea_world_labDEV JOURNAL
Thursday, June 25, 2026

Qwen PR Review Workflow Structure and Improvement Log

Date: June 25, 2026

Purpose

This document records the process of importing the qwen-code‑based PR review workflow and customizing it for the current repository.

The core of this work is not only to have the workflow finish successfully, but also to make the review results generated by Qwen appear as actual GitHub PR comments.

Background

This repository also uses external review bots such as CodeRabbit, Sourcery, and Qodo. However, we separately import the qwen-code workflow, run it via the OpenRouter route, and are experimenting with a flow that leaves a Qwen‑based review on each PR.

Initially, the AI PR review API flow was stopped due to cost concerns. Later, because the OpenRouter free tier allows the use of openai/gpt-oss-120b:free, the README was updated to try this route again.

.github/workflows/qwen-code-pr-review.yml

Original workflow name:

🧐 Qwen Pull Request Review

Main trigger:

on:
  pull_request_target:
    types: ['opened']
  pull_request_review_comment:
    types: ['created']
  pull_request_review:
    types: ['submitted']
  workflow_dispatch:
    inputs:
      pr_number:
        required: true
        type: 'number'

The job condition was set to run in the following cases.

  • Manual execution with workflow_dispatch
  • The opened event of pull_request_target when the PR author is an OWNER, MEMBER, or COLLABORATOR
  • When the PR review comment contains @qwen /review
  • When the PR review body contains @qwen /review

The main steps of the workflow were as follows.

  1. Checkout PR code
  2. Collect PR title, body, changed files, and diff information
  3. Run QwenLM/qwen-code-action
  4. In the prompt, instruct Qwen to create a review and post a comment with gh pr comment

Observed Issue

Observed the workflow in practice on PR #5.

PR:

https://github.com/yyeongjin/idea_world/pull/5

PR was created successfully and the following bots responded.

  • CodeRabbit
  • Sourcery
  • Qodo

The Qwen workflow also ran on the pull_request_target event.

Observed run:

workflow: 🧐 Qwen Pull Request Review
event: pull_request_target
conclusion: success

But in the actual PR, no Qwen review comment was left.

Looking at the workflow log, Qwen wrote the review content and then, instead of executing the actual tool call, output a JSON‑style instruction as plain text.

{"tool":"write_file", ...}
{"tool":"run_shell_command", ...}

In other words, the workflow itself was successful, but the expected result of the user, “writing a PR comment,” was not completed.

Cause Summary

The problem was that the structure delegated the comment writing to Qwen.

The original prompt instructed Qwen to perform the following directly.

1. write_file("review.md", "<your detailed review feedback here>")
2. gh pr comment $PR_NUMBER --body-file review.md --repo $REPOSITORY

But in actual execution, Qwen did not perform this task as a tool call, and output the required work only as text.

Therefore, if the model does not properly invoke the tool, even when the workflow succeeds, no comment will be left on the PR.

Additional Structural Issue Identified

The job condition of the workflow included the issue_comment event.

(github.event_name == 'issue_comment' && ... contains(github.event.comment.body, '@qwen /review') ...)

But the on: trigger did not include issue_comment.

In other words, even if you leave a comment @qwen /review in a PR conversation, the workflow would not start at all.

Also, the pull_request_target trigger only responded to opened.

pull_request_target:
  types: ['opened']

So the synchronize event, where a new commit is pushed to the PR branch, was not automatically re‑run.

Improvement Direction

Instead of having the model write GitHub comments, change it so that a GitHub Actions step is responsible for writing comments.

After the improvement, the responsibility separation is as follows.

Qwen
  -> Generate only review Markdown text

GitHub Actions
  -> Save Qwen's output to a file
  -> Post a comment to the PR using `gh pr comment`

If you do this, even when Qwen outputs tool calls as text, you can at least receive qwen_result in the workflow step and leave a comment.

Changes

1. Strengthen trigger

Original:

pull_request_target:
  types: ['opened']

Change:

pull_request_target:
  types: ['opened', 'reopened', 'synchronize']
issue_comment:
  types: ['created']

Intent:

  • Run when a PR is newly opened
  • Run when a PR is reopened
  • Run when a new commit is pushed to the PR branch
  • Run when @qwen /review is left in the PR conversation

Additional observations:

Just strengthening the on: trigger was not enough. In fact, when a new commit was pushed to the PR branch, a pull_request_target run was created, but the job condition still only allowed github.event.action == 'opened', so the run ended as skipped.

Therefore, the job condition must also be modified as follows.

(github.event.action == 'opened' ||
 github.event.action == 'reopened' ||
 github.event.action == 'synchronize')

2. Assign id to Qwen step

Change:

- name: 'Run Qwen PR Review'
  id: 'qwen_review'

Purpose:

It is for reading the following output in the subsequent step.

steps.qwen_review.outputs.qwen_result

3. Change of Prompt Responsibility

The original prompt directly instructed Qwen to create files and write PR comments.

After the change, the prompt instructs Qwen to return only the review Markdown text as the final response.

Return the review markdown text as your final answer.
Do not call write_file.
Do not call gh pr comment.
Do not output JSON tool-call instructions.
A later GitHub Actions step will post your final answer as a PR comment.

Purpose:

  • Reduce the issue of Qwen outputting tool‑call JSON.
  • Shift the responsibility for writing comments from the model to a workflow step.

4. Add a step to post Qwen’s output as a PR comment

Added step:

- name: 'Post Qwen PR Review'
  if: |-
    ${{ always() && steps.qwen_review.outputs.summary != '' }}
  env:
    GITHUB_TOKEN: '${{ secrets.GITHUB_TOKEN }}'
    PR_NUMBER: '${{ steps.get_pr.outputs.pr_number || steps.get_pr_comment.outputs.pr_number }}'
    REPOSITORY: '${{ github.repository }}'
    QWEN_RESULT: '${{ steps.qwen_review.outputs.summary }}'
  run: |-
    {
      echo "## Qwen PR Review"
      echo
      printf '%s\n' "$QWEN_RESULT"
    } > qwen-review.md
    gh pr comment "$PR_NUMBER" --body-file qwen-review.md --repo "$REPOSITORY"

Purpose:

  • If the Qwen output is not empty, always leave a PR comment.
  • Writing the comment is handled by a deterministic step that uses GITHUB_TOKEN and gh pr comment.

Additional observations:

At first, the output name was mistakenly read as steps.qwen_review.outputs.qwen_result. Checking the action.yml of QwenLM/qwen-code-action shows that the composite step creates qwen_result, but the output name exposed outside the action is summary.

outputs:
  summary:
    value: ${{ steps.qwen_run.outputs.qwen_result }}

So the subsequent comment step must read steps.qwen_review.outputs.summary. If you read qwen_result, even when Qwen generates the review body correctly, the post step will be skipped.

Things to Note

The pull_request_target workflow runs based on the workflow file in the base branch, not the workflow file in the PR branch.

Therefore, merely pushing a workflow change to the PR branch will not change the behavior of pull_request_target for the current PR.

To actually apply it, one of the following is required.

  1. Apply the workflow change to main first.
  2. Then leave @qwen /review on the PR or push a new commit to run the updated workflow.

In this task, we will first apply the workflow change to main, then trigger a Qwen review again on PR #5 to verify whether a comment is actually created.

Security Guidelines

Do not record sensitive values such as API keys, endpoint URLs, or provider base URLs in workflow documents.

In GitHub Actions logs, the following values must also be subject to secret masking.

OPENAI_API_KEY
OPENAI_BASE_URL
OPENAI_MODEL
GITHUB_TOKEN

The document only retains the fact of model changes and the workflow structure, without recording actual secret values.

Current Conclusion

What was confirmed through this observation is as follows.

  • The creation of the PR itself operated normally.
  • CodeRabbit, Sourcery, and Qodo responded to the PR correctly.
  • The Qwen workflow also finished with success on pull_request_target.
  • However, in the previous structure, Qwen could output tool invocation instructions as text without actually posting a comment.
  • Therefore, “workflow success” and “PR comment creation success” should be considered separately.
  • Changing the Qwen output so that a subsequent GitHub Actions step posts it directly as a comment is safer.