This website uses cookies to ensure you get the full experience. You can change this any time. Learn more

Bugs‌ ‌found‌ ‌in‌ CircleCI SaaS

› Bug Crawl›Bugs‌ ‌found‌ ‌in‌ CircleCI SaaS

CircleCI

CircleCI is a software delivery platform designed to streamline the process of deploying software for teams.

With seamless integration capabilities with popular services like AWS, Slack, and Atlassian, CircleCI enables users to easily connect their preferred tools. Its versatile architecture allows users to build customized workflows using the user-friendly API. The platform is distinguished by its FedRAMP certification and SOC 2 Type II compliance, ensuring security and reliability throughout the software delivery lifecycle.

During our in-depth QAwerk Bug Crawl, we discovered a number of issues that could potentially affect the platform’s functionality and user experience. Read on to learn more!

circleci.com

Error occurs upon clicking link in invited user's email

Severity:

Critical

Precondition:
  1. Create a mailbox that is different from the one associated with your CircleCI account.
  2. Log in to your CircleCI account.
  3. Add one project.
Steps to Reproduce:
  1. In the side menu, click on the “Dashboard” button.
  2. Click on the project name.
  3. Click on the “Add team members” button.
  4. Click on the “Invite” button.
  5. Enter the created email in the “Enter email(s) separated by space” field.
  6. Click on the “Select a role” drop-down list.
  7. Select “Org Admin.”
  8. Click on the “Send Invite(s)” button.
  9. Open the mailbox of the user to whom the invitation was sent.
  10. Open the letter you received.
  11. Click on the “Accept Invitation” button.
Environment:

Windows 10 Pro
Chrome v.118.0.5993.118, Firefox v.119.0 , Microsoft Edge v. 118.0.2088.76

Actual Result:

An error message, “This site can’t be reached,” occurs after clicking on a link from an invited user’s letter.

Expected Result:

No error should occur when following the link from the invitation letter.

Overlapping side menu icons in collapsed version

Severity:

Major

Precondition:

Log in to the account.

Steps to Reproduce:
  1. Scroll down the side menu.
  2. Click on the button to collapse the side menu.
Environment:

Windows 10 Pro
Chrome v.118.0.5993.118, Firefox v.119.0 , Microsoft Edge v. 118.0.2088.76

Actual Result:

The sidebar icons overlap each other in the collapsed version of the sidebar.

Expected Result:

When the sidebar is collapsed, the icons should not overlap each other.

Overlapping side menu icons in collapsed version
Overlapping side menu icons in collapsed version
Overlapping side menu icons in collapsed version

Clicking “Rerun Workflow from Start” button is delayed

Severity:

Major

Precondition:
  1. Log in to your CircleCI account.
  2. Add one project.
Steps to Reproduce:
  1. In the side menu, click on the “Dashboard” button.
  2. Click on the “Rerun workflow from start” button.
  3. Click on the “Rerun workflow from start” button again.
Environment:

Windows 10 Pro
Chrome v.118.0.5993.118, Firefox v.119.0 , Microsoft Edge v. 118.0.2088.76

Actual Result:

Clicking on the “Rerun workflow from start” button is delayed. This creates the impression that the user’s initial click did not register, prompting them to repeat the action multiple times, resulting in redundant actions.

Expected Result:

If the user clicks on the “Rerun workflow from start” button, the button should be disabled or blocked until the action is completed.

Terminal instruction code yields error

Severity:

Major

Precondition:

Log in to the CircleCI account.

Steps to Reproduce:
  1. Click on the “Projects” button in the side menu.
  2. Click on the “Create Project” button.
  3. Click on the “GitHub” button.
  4. Click on the copy button in the block labeled “1. Generate an SSH key pair in your terminal with the command below.”
  5. Type CMD in the computer search.
  6. Open a command prompt.
  7. Paste the copied content.
  8. Press “Enter” three times.
Environment:

Windows 10 Pro
Chrome v.118.0.5993.118, Firefox v.119.0 , Microsoft Edge v. 118.0.2088.76

Actual Result:

An error occurs when using the code from the instructions in the command prompt.

Expected Result:

The manual should contain up-to-date information and troubleshooting tips to address errors or issues that users may encounter.

Empty “Organization Plans” page

Severity:

Major

Precondition:

Log in to the CircleCI account.

Steps to Reproduce:
  1. Click on the “Profile” button at the bottom of the side menu.
  2. Click on the “Organization Plans” button.
Environment:

Windows 10 Pro
Chrome v.118.0.5993.118, Firefox v.119.0 , Microsoft Edge v. 118.0.2088.76

Actual Result:

The “Organization Plans” page does not contain any plan content.

Expected Result:

The “Organization Plans” page should contain information about the available plans.

Popup overlapped by project information header

Severity:

Minor

Precondition:
  1. Log in to the CircleCI account.
  2. Add one project.
Steps to Reproduce:
  1. In the side menu, click on the “Projects” button.
  2. Click on the three buttons next to the project.
  3. Click on the “People” button.
  4. Click on the “Assign Project Roles” button.
Environment:

Windows 10 Pro
Chrome v.118.0.5993.118, Firefox v.119.0 , Microsoft Edge v. 118.0.2088.76

Actual Result:

A project information header overlays the top of the popup.

Expected Result:

The header should not overlap the top of the popup.

Displaced “Success” status tick

Severity:

Minor

Precondition:
  1. Log in to the CircleCI account.
  2. Add one project.
Steps to Reproduce:
  1. In the side menu, click on the “Dashboard” button.
Environment:

Windows 10 Pro
Chrome v.118.0.5993.118, Firefox v.119.0 , Microsoft Edge v. 118.0.2088.76

Actual Result:

The “Success” status tick is shifted downward.

Expected Result:

The status tick should be aligned at the same height as the “Success” status.

Displaced “Success” status tick

“See More” button appears despite absence of additional content

Severity:

Minor

Precondition:
  1. Log in to the CircleCI account.
  2. Add one project.
Steps to Reproduce:
  1. Click on the “Releases” button in the side menu.
  2. Click on the “ENVIRONMENTS” button.
  3. Click the “Create Release Environment” button.
  4. In the drop-down list, select “Amazon SageMaker”.
  5. Fill in the “Name” field.
  6. Click the “Create Release Environment” button.
  7. Click the setup button.
  8. Click on the “Create New Token” button.
  9. Click on the “Create Token” button.
  10. Click the “Done” button.
  11. Click the “See more” button.
Environment:

Windows 10 Pro
Chrome v.118.0.5993.118, Firefox v.119.0 , Microsoft Edge v. 118.0.2088.76

Actual Result:

After adding the token, the “See more” button appears, even though there is no additional content.

Expected Result:

After adding a token, the “See more” button should not appear if there are no more tokens.

Notification indicator persists despite tab opening

Severity:

Minor

Precondition:
  1. Log in to the CircleCI account.
  2. Add one project.
Steps to Reproduce:
  1. Click on the “Releases” button in the side menu.
  2. Click on the “Dashboard” button in the side menu.
Environment:

Windows 10 Pro
Chrome v.118.0.5993.118, Firefox v.119.0 , Microsoft Edge v. 118.0.2088.76

Actual Result:

The circle indicating the notification does not disappear after opening the tab.

Expected Result:

The circle corresponding to the notification should disappear after opening the tab.

Lengthy title “Release Environment” alters layout

Severity:

Minor

Precondition:
  1. Log in to the CircleCI account.
  2. Add one project.
Steps to Reproduce:
  1. Click on the “Releases” button in the side menu.
  2. Click on the “ENVIRONMENTS” button.
  3. Click the “Create Release Environment” button.
  4. In the drop-down list, select “Amazon SageMaker”.
  5. Fill in the “Name” field.
  6. Click the “Create Release Environment” button.
  7. Click the setup button.
  8. Click the “Edit Environment” button.
  9. Click the “Create Release Environment” button.
  10. Enter more than 200 characters in the “New integration name:” field.
  11. Click the “Update” button.
Environment:

Windows 10 Pro
Chrome v.118.0.5993.118, Firefox v.119.0 , Microsoft Edge v. 118.0.2088.76

Actual Result:

The long name “Release Environment” is causing a layout shift.

Expected Result:

The long name “Release Environment” should not cause the layout to shift.

“Releases” button vanishes from side menu on “Projects” page navigation

Severity:

Minor

Precondition:

Log in to the account.

Steps to Reproduce:
  1. In the side menu, click on the “Dashboard” button.
  2. In the side menu, click on the “Projects” button.
Environment:

Windows 10 Pro
Chrome v.118.0.5993.118, Firefox v.119.0 , Microsoft Edge v. 118.0.2088.76

Actual Result:

The “Releases” button in the side menu disappears when the user is on the “Projects” page.

Expected Result:

The “Releases” button in the side menu should remain visible even when the user is on the “Projects” page.

Excessive scrolling in drop-down menu

Severity:

Trivial

Precondition:

Log in to the account.

Steps to Reproduce:
  1. In the side menu, click on the “Support” button.
Environment:

Windows 10 Pro
Chrome v.118.0.5993.118, Firefox v.119.0 , Microsoft Edge v. 118.0.2088.76

Actual Result:

The drop-down menu displays additional scrolling.

Expected Result:

The drop-down menu should not display scrolling if all content fits within the block.

Excessive scrolling in drop-down menu

Extra scrolls appear on new project addition page

Severity:

Trivial

Precondition:

Log in to the account.

Steps to Reproduce:
  1. In the side menu, click on the “Projects” button.
  2. Then, click on the “Create Project” button.
Environment:

Windows 10 Pro
Chrome v.118.0.5993.118, Firefox v.119.0 , Microsoft Edge v. 118.0.2088.76

Actual Result:

On the page for adding a new project, extra scrolls are displayed.

Expected Result:

Since all the content fits on the screen, scrolls should not be displayed.

Additional scroll appears on email confirmation page

Severity:

Trivial

Precondition:
  1. Create an account for the email address to which you have access.
  2. Log in to the account for the first time.
Steps to Reproduce:
  1. Click on the button to send a confirmation email for the account.
  2. Open your email inbox.
  3. Open the confirmation email.
  4. Click on the “Verify Email” button.
Environment:

Windows 10 Pro
Chrome v.118.0.5993.118, Firefox v.119.0 , Microsoft Edge v. 118.0.2088.76

Actual Result:

An additional scroll is displayed on the email confirmation page.

Expected Result:

The email confirmation page should not display unnecessary scrolling.

Blue frame appears around “Learn” block expansion button

Severity:

Trivial

Precondition:

Log in to the account.

Steps to Reproduce:
  1. In the side menu, click on the “Projects” button.
  2. Click on the button to expand the “Learn” block.
  3. Click on a part of the page outside the button.
Environment:

Windows 10 Pro
Chrome v.118.0.5993.118, Firefox v.119.0 , Microsoft Edge v. 118.0.2088.76

Actual Result:

When you click on the button to expand the “Learn” block, a blue frame is displayed around the button.

Expected Result:

When clicking on the button to expand the “Learn” block, the blue frame around the button should not be displayed.

During testing, I identified 15 bugs across different features and functionalities of the platform. These bugs varied from minor inconsistencies to critical issues affecting usability and performance. To uphold the reliability and effectiveness of CircleCI, I suggest conducting comprehensive regression testing to promptly resolve these bugs.
Iryna, QA engineer

Iryna, QA engineer

Need a reliable QA partner?

Hire us

RELATED BUG CRAWLS