Drieam - E-mail notifications and previews not functioning – Incident details

All systems operational

E-mail notifications and previews not functioning

Resolved
Degraded performance
Started over 1 year agoLasted about 2 hours

Affected

Portflow

Degraded performance from 10:45 AM to 12:49 PM

Updates
  • Resolved
    Resolved

    This incident has been resolved. All queues have been cleared and previews will be created instantly again.

    Post Mortem
    Due to an undocumented or improper documented change in an update in a package we use for background tasks, the production environment has suffered from a period of inactive tasks over the weekend. We have contacted the supplier to help prevent the issue in the future, we also have taken measures to ensure we are notified earlier on in the process. Our apologies for the inconvenience.

  • Monitoring
    Monitoring

    We have found the cause and have implemented a fix. We are currently monitoring the result.

    All functionality should be restored to normal behavior soon. Because of the backlog that was created, it might take a couple of hours for all functionality to be restored.

    Next update before: 17.00 CEST

  • Investigating
    Update

    As we are investigating the cause of the incident, we have identified that the following tasks are not being executed by Portflow:

    • File preview generation (for new uploads)
    • Notifications
    • Goal Timeline (new entries)
    • Exports

    We're continuing to investigate the cause and keep you updated.

    Next update before: 15.00 CEST

  • Investigating
    Investigating

    Currently, the job responsible for creating file previews and sending email invitations to external users is not functioning correctly. We are currently investigating this incident.

    Next update before: 14.00 CEST.