AM1 - Slowness in API Printing

Incident Report for BarTender Cloud Status Page

Postmortem

AM1 – BTC API Slowness in API Printing
Date: May 21, 2025
Time: 07:51 AM PDT – 08:06 AM PDT
Status: RESOLVED

Summary
On May 21, 2025, between 7:51 AM and 8:06 AM PDT, some customers in the AM1 region experienced slowness when printing through the API. The Print API remained available, but response times were temporarily degraded due to high system activity. The issue was quickly resolved, and performance has since returned to normal.

What Happened
A sudden and significant spike in incoming API requests temporarily overwhelmed the print job scheduler, a service responsible for processing print activities. This caused a slowdown in handling requests and delayed job execution for some users.

Resolution
To immediately address the issue, we increased the scheduler’s capacity and made infrastructure adjustments to separate critical tasks into dedicated environments. These changes helped reduce pressure on the system and restore normal performance.

Next Steps
To prevent similar issues in the future, we are:

  • A long-term fix is planned in an upcoming release: we will introduce a dedicated processing pool for print jobs. This will improve system reliability and performance by eliminating dependencies on shared scheduler capacity.
  • Enhancing our monitoring systems to better detect and respond to capacity issues in real time
  • Improving the way our infrastructure scales under high demand
  • Introducing predictive tools to help anticipate and avoid future slowdowns

We appreciate your understanding and remain committed to providing a stable and responsive service.

Posted May 21, 2025 - 17:07 PDT

Resolved

At 07:51 AM PDT, we observed intermittent slowness in the Print API in the AM1 region. There was no outage, but some customers experienced delays printing via the API.

The system has recovered at 08:06 AM PDT, and printing operations are now functioning normally. We appreciate your patience and understanding.
Posted May 21, 2025 - 07:51 PDT