Drieam - Some background tasks in Portflow are hanging (snapshots, template imports, notifications). – Incident details

Some background tasks in Portflow are hanging (snapshots, template imports, notifications).

Resolved
Degraded performance
Started 20 days agoLasted about 16 hours

Affected

Portflow

Degraded performance from 12:55 PM to 5:06 AM

Updates
  • Update
    Update

    This incident has been resolved.

    Since the fixes of early this morning, all background tasks are running smoothly and stable again. Our cloud hosting partner Heroku and SalesForce are implementing measures to prevent this from happening again.

  • Resolved
    Resolved
    The implemented fixes of Heroku last hours should have made some impact on alleviating the issues in Portflow. Simultaneously, we were able to get background tasks started up again with limited and somewhat unreliable resources. This should result in more tasks completing with a slight delay. We'll keep monitoring the situation. Next update before 10.00 CEST.
  • Update
    Update

    The Heroku and SalesForce team are working on a solution to mitigate the issue. The situation can be monitored on SalesForce's Trust Portal.

    Next update before: Tue 11 June 8.00 CEST.

  • Update
    Update

    Unfortunately, Heroku has not been able to implement a fix for this issue. We'll keep monitoring the situation.

    Next update before: 19.00 CEST.

  • Identified
    Identified

    We have identified the cause of a number of issues in Portflow at the moment.

    Our cloud hosting provider Heroku is experiencing outages which are causing background jobs that Portflow relies on to not complete. This may cause issues with snapshots not being finished, notifications not being sent or templates not being imported. We're closely monitoring the situation and will update you before 17.00 CEST.

  • Investigating
    Investigating

    Learners are experiencing that the snapshots they created are getting stuck in the creation process. We're currently investigating the issue.