Incidents reported on status page for unique.app https://status.prod.unique.app/ en https://status.prod.unique.app/ Sun, 15 Feb 2026 00:38:06 +0000 https://status.prod.unique.app/#b4ee8c6b55ae96064dd30330c7507c0904467a8d943579a0f3e2bf0b421bf267 Microsoft recovered https://status.prod.unique.app/ Sun, 15 Feb 2026 00:31:48 +0000 https://status.prod.unique.app/#26275776d2b4d23faf469b661b2f6b690d0163c6ec32a5c92b7939ba4725e618 Webhooks recovered https://status.prod.unique.app/ Sun, 15 Feb 2026 00:29:09 +0000 https://status.prod.unique.app/#b4ee8c6b55ae96064dd30330c7507c0904467a8d943579a0f3e2bf0b421bf267 Microsoft went down https://status.prod.unique.app/ Sun, 15 Feb 2026 00:28:53 +0000 https://status.prod.unique.app/#26275776d2b4d23faf469b661b2f6b690d0163c6ec32a5c92b7939ba4725e618 Webhooks went down https://status.prod.unique.app/ Mon, 09 Feb 2026 12:53:00 +0000 https://status.prod.unique.app/#0d5554815bb7e61004761defb7747fb7805fd78d6dc366456bbba59420c372d8 Webhooks recovered https://status.prod.unique.app/ Mon, 09 Feb 2026 12:49:55 +0000 https://status.prod.unique.app/#0d5554815bb7e61004761defb7747fb7805fd78d6dc366456bbba59420c372d8 Webhooks went down https://status.prod.unique.app/ Mon, 09 Feb 2026 03:36:31 +0000 https://status.prod.unique.app/#6a8aba81f4acaed087a88e46754f04f872a71fbb33bbf5670e33b9b580ca4dbf Sidebar recovered https://status.prod.unique.app/ Mon, 09 Feb 2026 03:33:16 +0000 https://status.prod.unique.app/#6a8aba81f4acaed087a88e46754f04f872a71fbb33bbf5670e33b9b580ca4dbf Sidebar went down https://status.prod.unique.app/ Mon, 02 Feb 2026 03:32:23 +0000 https://status.prod.unique.app/#896d09c91717158a0ed88529ba22cd598f47f0c10f732b2fb93e3ff485db43f1 Microsoft recovered https://status.prod.unique.app/ Mon, 02 Feb 2026 03:29:16 +0000 https://status.prod.unique.app/#896d09c91717158a0ed88529ba22cd598f47f0c10f732b2fb93e3ff485db43f1 Microsoft went down https://status.prod.unique.app/ Thu, 29 Jan 2026 07:31:59 +0000 https://status.prod.unique.app/#91aa99e5a98674321593b715d82bdeee88891b377589575953c8bb5b262abfde Webhooks recovered https://status.prod.unique.app/ Thu, 29 Jan 2026 07:28:59 +0000 https://status.prod.unique.app/#91aa99e5a98674321593b715d82bdeee88891b377589575953c8bb5b262abfde Webhooks went down https://status.prod.unique.app/ Thu, 29 Jan 2026 07:07:51 +0000 https://status.prod.unique.app/#4dff644f7088052139ea9ec114bf4a0ea98561895dc4f38c947634a497aa94a7 Webhooks recovered https://status.prod.unique.app/ Thu, 29 Jan 2026 07:05:53 +0000 https://status.prod.unique.app/#4dff644f7088052139ea9ec114bf4a0ea98561895dc4f38c947634a497aa94a7 Webhooks went down https://status.prod.unique.app/incident/812551 Thu, 29 Jan 2026 07:00:08 -0000 https://status.prod.unique.app/incident/812551#d669255b438e5fc5ac40e6015f3c70c02ad142c17ad43132107a50b448fc7c5b Maintenance completed https://status.prod.unique.app/incident/812551 Thu, 29 Jan 2026 07:00:08 -0000 https://status.prod.unique.app/incident/812551#d669255b438e5fc5ac40e6015f3c70c02ad142c17ad43132107a50b448fc7c5b Maintenance completed https://status.prod.unique.app/incident/812551 Thu, 29 Jan 2026 07:00:08 -0000 https://status.prod.unique.app/incident/812551#d669255b438e5fc5ac40e6015f3c70c02ad142c17ad43132107a50b448fc7c5b Maintenance completed https://status.prod.unique.app/incident/812551 Thu, 29 Jan 2026 07:00:08 -0000 https://status.prod.unique.app/incident/812551#d669255b438e5fc5ac40e6015f3c70c02ad142c17ad43132107a50b448fc7c5b Maintenance completed https://status.prod.unique.app/incident/812551 Thu, 29 Jan 2026 07:00:08 -0000 https://status.prod.unique.app/incident/812551#d669255b438e5fc5ac40e6015f3c70c02ad142c17ad43132107a50b448fc7c5b Maintenance completed https://status.prod.unique.app/incident/812551 Thu, 29 Jan 2026 07:00:08 -0000 https://status.prod.unique.app/incident/812551#d669255b438e5fc5ac40e6015f3c70c02ad142c17ad43132107a50b448fc7c5b Maintenance completed https://status.prod.unique.app/incident/812551 Thu, 29 Jan 2026 07:00:08 -0000 https://status.prod.unique.app/incident/812551#d669255b438e5fc5ac40e6015f3c70c02ad142c17ad43132107a50b448fc7c5b Maintenance completed https://status.prod.unique.app/incident/812551 Thu, 29 Jan 2026 07:00:08 -0000 https://status.prod.unique.app/incident/812551#d669255b438e5fc5ac40e6015f3c70c02ad142c17ad43132107a50b448fc7c5b Maintenance completed https://status.prod.unique.app/incident/812551 Thu, 29 Jan 2026 07:00:08 -0000 https://status.prod.unique.app/incident/812551#d669255b438e5fc5ac40e6015f3c70c02ad142c17ad43132107a50b448fc7c5b Maintenance completed https://status.prod.unique.app/incident/812551 Thu, 29 Jan 2026 07:00:08 -0000 https://status.prod.unique.app/incident/812551#d669255b438e5fc5ac40e6015f3c70c02ad142c17ad43132107a50b448fc7c5b Maintenance completed https://status.prod.unique.app/incident/812551 Thu, 29 Jan 2026 07:00:08 -0000 https://status.prod.unique.app/incident/812551#d669255b438e5fc5ac40e6015f3c70c02ad142c17ad43132107a50b448fc7c5b Maintenance completed https://status.prod.unique.app/ Thu, 29 Jan 2026 06:42:11 +0000 https://status.prod.unique.app/#ea30709226d926a6086a8e4f21a022820ed523c79cd93de94dbfed09933846cd Sidebar recovered https://status.prod.unique.app/ Thu, 29 Jan 2026 06:39:17 +0000 https://status.prod.unique.app/#ea30709226d926a6086a8e4f21a022820ed523c79cd93de94dbfed09933846cd Sidebar went down https://status.prod.unique.app/ Thu, 29 Jan 2026 06:12:15 +0000 https://status.prod.unique.app/#bb25673cb06db7c2906e7be2234e2e3f4188afa736de04083914b2ce4b94f90a Sidebar recovered https://status.prod.unique.app/ Thu, 29 Jan 2026 06:09:11 +0000 https://status.prod.unique.app/#bb25673cb06db7c2906e7be2234e2e3f4188afa736de04083914b2ce4b94f90a Sidebar went down https://status.prod.unique.app/ Thu, 29 Jan 2026 06:05:11 +0000 https://status.prod.unique.app/#89ef689ecd0bedb440a055af646c1074aa2500fa1302d6c28d562ddc1f689bb5 Webhooks recovered https://status.prod.unique.app/ Thu, 29 Jan 2026 06:05:10 +0000 https://status.prod.unique.app/#ec5d2845c15eb4a26fcb9ceb115bcc2dc5167abe4591a64d07d71c3599a84edf Scope Management recovered https://status.prod.unique.app/ Thu, 29 Jan 2026 06:02:20 +0000 https://status.prod.unique.app/#89ef689ecd0bedb440a055af646c1074aa2500fa1302d6c28d562ddc1f689bb5 Webhooks went down https://status.prod.unique.app/ Thu, 29 Jan 2026 06:02:05 +0000 https://status.prod.unique.app/#ec5d2845c15eb4a26fcb9ceb115bcc2dc5167abe4591a64d07d71c3599a84edf Scope Management went down https://status.prod.unique.app/ Thu, 29 Jan 2026 05:39:17 +0000 https://status.prod.unique.app/#fe430b039a18a756b166ab6207ac52a21290b1d83d0b050a00dc7ddbdc5471eb Sidebar recovered https://status.prod.unique.app/ Thu, 29 Jan 2026 05:38:17 +0000 https://status.prod.unique.app/#dcca657ee9d768117cdc80e16a6fc17f5ee7768153210f361cffe651ee729d4c Microsoft recovered https://status.prod.unique.app/ Thu, 29 Jan 2026 05:36:14 +0000 https://status.prod.unique.app/#fe430b039a18a756b166ab6207ac52a21290b1d83d0b050a00dc7ddbdc5471eb Sidebar went down https://status.prod.unique.app/ Thu, 29 Jan 2026 05:35:16 +0000 https://status.prod.unique.app/#dcca657ee9d768117cdc80e16a6fc17f5ee7768153210f361cffe651ee729d4c Microsoft went down https://status.prod.unique.app/incident/812551 Thu, 29 Jan 2026 04:00:08 -0000 https://status.prod.unique.app/incident/812551#ded890ac7ebc597e7c98744203d28997600d6415b6161d0cc0adc91a9e45439c The Kubernetes cluster will be migrated to Cilium CNI. There may be occasional short interruptions. https://status.prod.unique.app/incident/812551 Thu, 29 Jan 2026 04:00:08 -0000 https://status.prod.unique.app/incident/812551#ded890ac7ebc597e7c98744203d28997600d6415b6161d0cc0adc91a9e45439c The Kubernetes cluster will be migrated to Cilium CNI. There may be occasional short interruptions. https://status.prod.unique.app/incident/812551 Thu, 29 Jan 2026 04:00:08 -0000 https://status.prod.unique.app/incident/812551#ded890ac7ebc597e7c98744203d28997600d6415b6161d0cc0adc91a9e45439c The Kubernetes cluster will be migrated to Cilium CNI. There may be occasional short interruptions. https://status.prod.unique.app/incident/812551 Thu, 29 Jan 2026 04:00:08 -0000 https://status.prod.unique.app/incident/812551#ded890ac7ebc597e7c98744203d28997600d6415b6161d0cc0adc91a9e45439c The Kubernetes cluster will be migrated to Cilium CNI. There may be occasional short interruptions. https://status.prod.unique.app/incident/812551 Thu, 29 Jan 2026 04:00:08 -0000 https://status.prod.unique.app/incident/812551#ded890ac7ebc597e7c98744203d28997600d6415b6161d0cc0adc91a9e45439c The Kubernetes cluster will be migrated to Cilium CNI. There may be occasional short interruptions. https://status.prod.unique.app/incident/812551 Thu, 29 Jan 2026 04:00:08 -0000 https://status.prod.unique.app/incident/812551#ded890ac7ebc597e7c98744203d28997600d6415b6161d0cc0adc91a9e45439c The Kubernetes cluster will be migrated to Cilium CNI. There may be occasional short interruptions. https://status.prod.unique.app/incident/812551 Thu, 29 Jan 2026 04:00:08 -0000 https://status.prod.unique.app/incident/812551#ded890ac7ebc597e7c98744203d28997600d6415b6161d0cc0adc91a9e45439c The Kubernetes cluster will be migrated to Cilium CNI. There may be occasional short interruptions. https://status.prod.unique.app/incident/812551 Thu, 29 Jan 2026 04:00:08 -0000 https://status.prod.unique.app/incident/812551#ded890ac7ebc597e7c98744203d28997600d6415b6161d0cc0adc91a9e45439c The Kubernetes cluster will be migrated to Cilium CNI. There may be occasional short interruptions. https://status.prod.unique.app/incident/812551 Thu, 29 Jan 2026 04:00:08 -0000 https://status.prod.unique.app/incident/812551#ded890ac7ebc597e7c98744203d28997600d6415b6161d0cc0adc91a9e45439c The Kubernetes cluster will be migrated to Cilium CNI. There may be occasional short interruptions. https://status.prod.unique.app/incident/812551 Thu, 29 Jan 2026 04:00:08 -0000 https://status.prod.unique.app/incident/812551#ded890ac7ebc597e7c98744203d28997600d6415b6161d0cc0adc91a9e45439c The Kubernetes cluster will be migrated to Cilium CNI. There may be occasional short interruptions. https://status.prod.unique.app/incident/812551 Thu, 29 Jan 2026 04:00:08 -0000 https://status.prod.unique.app/incident/812551#ded890ac7ebc597e7c98744203d28997600d6415b6161d0cc0adc91a9e45439c The Kubernetes cluster will be migrated to Cilium CNI. There may be occasional short interruptions. https://status.prod.unique.app/ Mon, 26 Jan 2026 10:20:20 +0000 https://status.prod.unique.app/#d0140bd26e78740b7e40069bfe4ad1b6c8cbf418a3ed15e97c13c0f17bae5aa4 Microsoft recovered https://status.prod.unique.app/ Mon, 26 Jan 2026 10:17:42 +0000 https://status.prod.unique.app/#d0140bd26e78740b7e40069bfe4ad1b6c8cbf418a3ed15e97c13c0f17bae5aa4 Microsoft went down https://status.prod.unique.app/ Mon, 26 Jan 2026 08:26:13 +0000 https://status.prod.unique.app/#ceacd6e945ef2c0f23afa1742eb8a7cecef21938def4531c7cce6d8cc3d8e6d0 Microsoft recovered https://status.prod.unique.app/ Mon, 26 Jan 2026 08:23:22 +0000 https://status.prod.unique.app/#ceacd6e945ef2c0f23afa1742eb8a7cecef21938def4531c7cce6d8cc3d8e6d0 Microsoft went down https://status.prod.unique.app/ Sun, 25 Jan 2026 00:07:53 +0000 https://status.prod.unique.app/#814dfee9a896c10c44a4842de320bf3c42315dbe4553c2d0aac51b11b3588c48 Webhooks recovered https://status.prod.unique.app/ Sun, 25 Jan 2026 00:04:54 +0000 https://status.prod.unique.app/#814dfee9a896c10c44a4842de320bf3c42315dbe4553c2d0aac51b11b3588c48 Webhooks went down https://status.prod.unique.app/ Fri, 23 Jan 2026 22:26:49 +0000 https://status.prod.unique.app/#91d7d1e57fd69706f640b5929beebcf4f0b832acf2cf09a29dcc15e45ae3be1c IDP recovered https://status.prod.unique.app/ Fri, 23 Jan 2026 22:23:50 +0000 https://status.prod.unique.app/#91d7d1e57fd69706f640b5929beebcf4f0b832acf2cf09a29dcc15e45ae3be1c IDP went down https://status.prod.unique.app/ Fri, 23 Jan 2026 09:08:15 +0000 https://status.prod.unique.app/#b59d343ebb03599920e3785dae52ccdbc9fbb14b52375597bd72792c1482af2a Microsoft recovered https://status.prod.unique.app/ Fri, 23 Jan 2026 09:05:12 +0000 https://status.prod.unique.app/#b59d343ebb03599920e3785dae52ccdbc9fbb14b52375597bd72792c1482af2a Microsoft went down https://status.prod.unique.app/ Thu, 22 Jan 2026 16:24:16 +0000 https://status.prod.unique.app/#b49c5610c01223a1761a42cf9d728464178461485a9332b3131872ba71a70a48 Sidebar recovered https://status.prod.unique.app/ Thu, 22 Jan 2026 16:21:07 +0000 https://status.prod.unique.app/#b49c5610c01223a1761a42cf9d728464178461485a9332b3131872ba71a70a48 Sidebar went down https://status.prod.unique.app/ Mon, 19 Jan 2026 23:18:27 +0000 https://status.prod.unique.app/#271ecf245692593ee04a23a771c09e4a5a16953d6feb6db028efed6afeefa229 Chat recovered https://status.prod.unique.app/ Mon, 19 Jan 2026 23:15:42 +0000 https://status.prod.unique.app/#271ecf245692593ee04a23a771c09e4a5a16953d6feb6db028efed6afeefa229 Chat went down https://status.prod.unique.app/ Sun, 18 Jan 2026 02:23:01 +0000 https://status.prod.unique.app/#1b685641a80eee1d9218932165b145999606769ba4e6a83825ba521d9f817c00 Scope Management recovered https://status.prod.unique.app/ Sun, 18 Jan 2026 02:20:01 +0000 https://status.prod.unique.app/#1b685641a80eee1d9218932165b145999606769ba4e6a83825ba521d9f817c00 Scope Management went down https://status.prod.unique.app/ Sun, 18 Jan 2026 02:02:12 +0000 https://status.prod.unique.app/#47abbdbd9226f13acfc430ee6040937465ce99e3668d37b6e45e188890396fa9 Microsoft recovered https://status.prod.unique.app/ Sun, 18 Jan 2026 01:59:40 +0000 https://status.prod.unique.app/#47abbdbd9226f13acfc430ee6040937465ce99e3668d37b6e45e188890396fa9 Microsoft went down https://status.prod.unique.app/ Sun, 18 Jan 2026 01:49:51 +0000 https://status.prod.unique.app/#90efbe56d40d1801cbe1f5ab421a458b2d6a49f25b7a8c43b3a71ac8029aafd7 Webhooks recovered https://status.prod.unique.app/ Sun, 18 Jan 2026 01:46:51 +0000 https://status.prod.unique.app/#90efbe56d40d1801cbe1f5ab421a458b2d6a49f25b7a8c43b3a71ac8029aafd7 Webhooks went down https://status.prod.unique.app/ Wed, 14 Jan 2026 20:39:17 +0000 https://status.prod.unique.app/#e7317c3c1a6d3af6560d7419181f0f1779d8d2a456966e800aac8076721fa4df Sidebar recovered https://status.prod.unique.app/ Wed, 14 Jan 2026 20:36:23 +0000 https://status.prod.unique.app/#e7317c3c1a6d3af6560d7419181f0f1779d8d2a456966e800aac8076721fa4df Sidebar went down https://status.prod.unique.app/ Wed, 14 Jan 2026 19:26:11 +0000 https://status.prod.unique.app/#7e91bb324f34f772633902358523f4f7c9509a6d138ea8ececfbccb816217cff Microsoft recovered https://status.prod.unique.app/ Wed, 14 Jan 2026 19:25:54 +0000 https://status.prod.unique.app/#fe0e2de1c9fa6d4d315bf66e08bdbefe5c4e2760f43b2ab811293803c5792a65 Webhooks recovered https://status.prod.unique.app/ Wed, 14 Jan 2026 19:23:15 +0000 https://status.prod.unique.app/#7e91bb324f34f772633902358523f4f7c9509a6d138ea8ececfbccb816217cff Microsoft went down https://status.prod.unique.app/ Wed, 14 Jan 2026 19:23:07 +0000 https://status.prod.unique.app/#fe0e2de1c9fa6d4d315bf66e08bdbefe5c4e2760f43b2ab811293803c5792a65 Webhooks went down https://status.prod.unique.app/ Mon, 05 Jan 2026 09:15:49 +0000 https://status.prod.unique.app/#1d7225d6dea38be73c49178ac6e7468a899c001a8c089cbca25138f0d8dd588e Chat recovered https://status.prod.unique.app/ Mon, 05 Jan 2026 09:12:30 +0000 https://status.prod.unique.app/#1d7225d6dea38be73c49178ac6e7468a899c001a8c089cbca25138f0d8dd588e Chat went down https://status.prod.unique.app/incident/796523 Sun, 04 Jan 2026 00:00:00 -0000 https://status.prod.unique.app/incident/796523#fa10a30eeddc0c22f516ec26efd2450790cb04320d281d351830e2bd3c814fae Maintenance completed https://status.prod.unique.app/incident/796523 Sun, 04 Jan 2026 00:00:00 -0000 https://status.prod.unique.app/incident/796523#fa10a30eeddc0c22f516ec26efd2450790cb04320d281d351830e2bd3c814fae Maintenance completed https://status.prod.unique.app/incident/796523 Sun, 04 Jan 2026 00:00:00 -0000 https://status.prod.unique.app/incident/796523#fa10a30eeddc0c22f516ec26efd2450790cb04320d281d351830e2bd3c814fae Maintenance completed https://status.prod.unique.app/incident/796523 Sun, 04 Jan 2026 00:00:00 -0000 https://status.prod.unique.app/incident/796523#fa10a30eeddc0c22f516ec26efd2450790cb04320d281d351830e2bd3c814fae Maintenance completed https://status.prod.unique.app/incident/796523 Sun, 04 Jan 2026 00:00:00 -0000 https://status.prod.unique.app/incident/796523#fa10a30eeddc0c22f516ec26efd2450790cb04320d281d351830e2bd3c814fae Maintenance completed https://status.prod.unique.app/incident/796523 Sun, 04 Jan 2026 00:00:00 -0000 https://status.prod.unique.app/incident/796523#fa10a30eeddc0c22f516ec26efd2450790cb04320d281d351830e2bd3c814fae Maintenance completed https://status.prod.unique.app/incident/796523 Sun, 04 Jan 2026 00:00:00 -0000 https://status.prod.unique.app/incident/796523#fa10a30eeddc0c22f516ec26efd2450790cb04320d281d351830e2bd3c814fae Maintenance completed https://status.prod.unique.app/incident/796523 Sun, 04 Jan 2026 00:00:00 -0000 https://status.prod.unique.app/incident/796523#fa10a30eeddc0c22f516ec26efd2450790cb04320d281d351830e2bd3c814fae Maintenance completed https://status.prod.unique.app/incident/796523 Sun, 04 Jan 2026 00:00:00 -0000 https://status.prod.unique.app/incident/796523#fa10a30eeddc0c22f516ec26efd2450790cb04320d281d351830e2bd3c814fae Maintenance completed https://status.prod.unique.app/incident/796523 Sun, 04 Jan 2026 00:00:00 -0000 https://status.prod.unique.app/incident/796523#fa10a30eeddc0c22f516ec26efd2450790cb04320d281d351830e2bd3c814fae Maintenance completed https://status.prod.unique.app/incident/796523 Sun, 04 Jan 2026 00:00:00 -0000 https://status.prod.unique.app/incident/796523#fa10a30eeddc0c22f516ec26efd2450790cb04320d281d351830e2bd3c814fae Maintenance completed https://status.prod.unique.app/incident/796523 Sun, 04 Jan 2026 00:00:00 -0000 https://status.prod.unique.app/incident/796523#fa10a30eeddc0c22f516ec26efd2450790cb04320d281d351830e2bd3c814fae Maintenance completed https://status.prod.unique.app/ Sat, 03 Jan 2026 20:38:23 +0000 https://status.prod.unique.app/#8f29e25b08cd2008a89e1a9d3efd0f8ac795f79c1f60d6802329b2ff330f902b Microsoft recovered https://status.prod.unique.app/ Sat, 03 Jan 2026 20:35:14 +0000 https://status.prod.unique.app/#8f29e25b08cd2008a89e1a9d3efd0f8ac795f79c1f60d6802329b2ff330f902b Microsoft went down https://status.prod.unique.app/ Sat, 03 Jan 2026 20:27:13 +0000 https://status.prod.unique.app/#a141f202abd7c8d8bc02086714574c350b93fc2c880f234aa4b9e0d4d60d76c0 Sidebar recovered https://status.prod.unique.app/ Sat, 03 Jan 2026 20:26:05 +0000 https://status.prod.unique.app/#66eba20eaeef8c868947bd0946a90a3a2eca932dcfb51b98af453ab462e8b54f Scope Management recovered https://status.prod.unique.app/ Sat, 03 Jan 2026 20:25:54 +0000 https://status.prod.unique.app/#a119c7f90dffef8c7b206d4b7a346a26397fb486a758742cdfc748eaece7d4a9 Webhooks recovered https://status.prod.unique.app/ Sat, 03 Jan 2026 20:23:08 +0000 https://status.prod.unique.app/#66eba20eaeef8c868947bd0946a90a3a2eca932dcfb51b98af453ab462e8b54f Scope Management went down https://status.prod.unique.app/ Sat, 03 Jan 2026 20:22:50 +0000 https://status.prod.unique.app/#a119c7f90dffef8c7b206d4b7a346a26397fb486a758742cdfc748eaece7d4a9 Webhooks went down https://status.prod.unique.app/ Sat, 03 Jan 2026 20:21:13 +0000 https://status.prod.unique.app/#a141f202abd7c8d8bc02086714574c350b93fc2c880f234aa4b9e0d4d60d76c0 Sidebar went down https://status.prod.unique.app/ Sat, 03 Jan 2026 20:20:13 +0000 https://status.prod.unique.app/#9f0e608bdf49bfd380a13f389160e2c926d4654fe9508d4b0a056f13b774f310 Microsoft recovered https://status.prod.unique.app/ Sat, 03 Jan 2026 20:17:16 +0000 https://status.prod.unique.app/#9f0e608bdf49bfd380a13f389160e2c926d4654fe9508d4b0a056f13b774f310 Microsoft went down https://status.prod.unique.app/ Sat, 03 Jan 2026 19:55:57 +0000 https://status.prod.unique.app/#912c3f2655c1c3ab43ba7201aba9f09283186d111ac2c119348059cb61772c17 Webhooks recovered https://status.prod.unique.app/ Sat, 03 Jan 2026 19:54:12 +0000 https://status.prod.unique.app/#a2b0d81a46cacb81791e3bd9dddc64c629ff1175e9b75bc741a4b165b0ab43a9 Sidebar recovered https://status.prod.unique.app/ Sat, 03 Jan 2026 19:51:18 +0000 https://status.prod.unique.app/#a2b0d81a46cacb81791e3bd9dddc64c629ff1175e9b75bc741a4b165b0ab43a9 Sidebar went down https://status.prod.unique.app/ Sat, 03 Jan 2026 19:50:19 +0000 https://status.prod.unique.app/#61a618bb2c1cc9bd47013792f2bcec3f1ee2f189c4d216578af02c6e990bc452 Microsoft recovered https://status.prod.unique.app/ Sat, 03 Jan 2026 19:49:55 +0000 https://status.prod.unique.app/#912c3f2655c1c3ab43ba7201aba9f09283186d111ac2c119348059cb61772c17 Webhooks went down https://status.prod.unique.app/ Sat, 03 Jan 2026 19:47:16 +0000 https://status.prod.unique.app/#61a618bb2c1cc9bd47013792f2bcec3f1ee2f189c4d216578af02c6e990bc452 Microsoft went down https://status.prod.unique.app/ Sat, 03 Jan 2026 19:25:57 +0000 https://status.prod.unique.app/#784439062bad5751616a6a5edbbfd63daa201572132e335d57a05908db40a998 Webhooks recovered https://status.prod.unique.app/ Sat, 03 Jan 2026 19:24:10 +0000 https://status.prod.unique.app/#83b0732c2f92224aaec2359448be86c38f4fbd82b3100bf5b5dbfe4e38eb33e4 Sidebar recovered https://status.prod.unique.app/ Sat, 03 Jan 2026 19:22:56 +0000 https://status.prod.unique.app/#784439062bad5751616a6a5edbbfd63daa201572132e335d57a05908db40a998 Webhooks went down https://status.prod.unique.app/ Sat, 03 Jan 2026 19:21:08 +0000 https://status.prod.unique.app/#83b0732c2f92224aaec2359448be86c38f4fbd82b3100bf5b5dbfe4e38eb33e4 Sidebar went down https://status.prod.unique.app/ Sat, 03 Jan 2026 19:06:36 +0000 https://status.prod.unique.app/#95bc9f4296d54ee8ecc4ef49de4d2a39b1cb1747a6732d6608016dd880d6e56d Sidebar recovered https://status.prod.unique.app/ Sat, 03 Jan 2026 19:05:14 +0000 https://status.prod.unique.app/#b3624df89bbee315b31dcd90315d1b3b6052b1c81a9e32805153e1fb497ad903 Microsoft recovered https://status.prod.unique.app/ Sat, 03 Jan 2026 19:04:56 +0000 https://status.prod.unique.app/#5c1e9a725cc505da46bdeb4e33bec613843b1ebe327b3227e2ad9e508eedb13c Webhooks recovered https://status.prod.unique.app/ Sat, 03 Jan 2026 19:03:20 +0000 https://status.prod.unique.app/#95bc9f4296d54ee8ecc4ef49de4d2a39b1cb1747a6732d6608016dd880d6e56d Sidebar went down https://status.prod.unique.app/ Sat, 03 Jan 2026 19:02:15 +0000 https://status.prod.unique.app/#b3624df89bbee315b31dcd90315d1b3b6052b1c81a9e32805153e1fb497ad903 Microsoft went down https://status.prod.unique.app/ Sat, 03 Jan 2026 19:01:57 +0000 https://status.prod.unique.app/#5c1e9a725cc505da46bdeb4e33bec613843b1ebe327b3227e2ad9e508eedb13c Webhooks went down https://status.prod.unique.app/incident/796523 Sat, 03 Jan 2026 18:00:00 -0000 https://status.prod.unique.app/incident/796523#44974050ca5d8d58e275e46f0bb30e6d307e0277b2f71cf87ca2a3da64d8d5ff Kubernetes cluster upgrade to the latest version. Service interruptions are expected during this maintenance window. https://status.prod.unique.app/incident/796523 Sat, 03 Jan 2026 18:00:00 -0000 https://status.prod.unique.app/incident/796523#44974050ca5d8d58e275e46f0bb30e6d307e0277b2f71cf87ca2a3da64d8d5ff Kubernetes cluster upgrade to the latest version. Service interruptions are expected during this maintenance window. https://status.prod.unique.app/incident/796523 Sat, 03 Jan 2026 18:00:00 -0000 https://status.prod.unique.app/incident/796523#44974050ca5d8d58e275e46f0bb30e6d307e0277b2f71cf87ca2a3da64d8d5ff Kubernetes cluster upgrade to the latest version. Service interruptions are expected during this maintenance window. https://status.prod.unique.app/incident/796523 Sat, 03 Jan 2026 18:00:00 -0000 https://status.prod.unique.app/incident/796523#44974050ca5d8d58e275e46f0bb30e6d307e0277b2f71cf87ca2a3da64d8d5ff Kubernetes cluster upgrade to the latest version. Service interruptions are expected during this maintenance window. https://status.prod.unique.app/incident/796523 Sat, 03 Jan 2026 18:00:00 -0000 https://status.prod.unique.app/incident/796523#44974050ca5d8d58e275e46f0bb30e6d307e0277b2f71cf87ca2a3da64d8d5ff Kubernetes cluster upgrade to the latest version. Service interruptions are expected during this maintenance window. https://status.prod.unique.app/incident/796523 Sat, 03 Jan 2026 18:00:00 -0000 https://status.prod.unique.app/incident/796523#44974050ca5d8d58e275e46f0bb30e6d307e0277b2f71cf87ca2a3da64d8d5ff Kubernetes cluster upgrade to the latest version. Service interruptions are expected during this maintenance window. https://status.prod.unique.app/incident/796523 Sat, 03 Jan 2026 18:00:00 -0000 https://status.prod.unique.app/incident/796523#44974050ca5d8d58e275e46f0bb30e6d307e0277b2f71cf87ca2a3da64d8d5ff Kubernetes cluster upgrade to the latest version. Service interruptions are expected during this maintenance window. https://status.prod.unique.app/incident/796523 Sat, 03 Jan 2026 18:00:00 -0000 https://status.prod.unique.app/incident/796523#44974050ca5d8d58e275e46f0bb30e6d307e0277b2f71cf87ca2a3da64d8d5ff Kubernetes cluster upgrade to the latest version. Service interruptions are expected during this maintenance window. https://status.prod.unique.app/incident/796523 Sat, 03 Jan 2026 18:00:00 -0000 https://status.prod.unique.app/incident/796523#44974050ca5d8d58e275e46f0bb30e6d307e0277b2f71cf87ca2a3da64d8d5ff Kubernetes cluster upgrade to the latest version. Service interruptions are expected during this maintenance window. https://status.prod.unique.app/incident/796523 Sat, 03 Jan 2026 18:00:00 -0000 https://status.prod.unique.app/incident/796523#44974050ca5d8d58e275e46f0bb30e6d307e0277b2f71cf87ca2a3da64d8d5ff Kubernetes cluster upgrade to the latest version. Service interruptions are expected during this maintenance window. https://status.prod.unique.app/incident/796523 Sat, 03 Jan 2026 18:00:00 -0000 https://status.prod.unique.app/incident/796523#44974050ca5d8d58e275e46f0bb30e6d307e0277b2f71cf87ca2a3da64d8d5ff Kubernetes cluster upgrade to the latest version. Service interruptions are expected during this maintenance window. https://status.prod.unique.app/incident/796523 Sat, 03 Jan 2026 18:00:00 -0000 https://status.prod.unique.app/incident/796523#44974050ca5d8d58e275e46f0bb30e6d307e0277b2f71cf87ca2a3da64d8d5ff Kubernetes cluster upgrade to the latest version. Service interruptions are expected during this maintenance window. https://status.prod.unique.app/ Thu, 18 Dec 2025 08:38:02 +0000 https://status.prod.unique.app/#60eb3954014765e06d6aa2c1f459e4a0cb566c0ba9fc4064e5b6d9f093f4190c Scope Management recovered https://status.prod.unique.app/ Thu, 18 Dec 2025 08:35:05 +0000 https://status.prod.unique.app/#60eb3954014765e06d6aa2c1f459e4a0cb566c0ba9fc4064e5b6d9f093f4190c Scope Management went down https://status.prod.unique.app/ Tue, 16 Dec 2025 10:39:56 +0000 https://status.prod.unique.app/#ac618714db0b40ddfe37f198940d48c1a762200ec9f1c0e952d3c22225e98f06 Knowledge Base recovered https://status.prod.unique.app/ Tue, 16 Dec 2025 10:39:54 +0000 https://status.prod.unique.app/#23b02dcb33fc37660e7e7999d2b595fe4c416adeb2286be9887a57b03e70f392 Recording recovered https://status.prod.unique.app/ Tue, 16 Dec 2025 10:39:47 +0000 https://status.prod.unique.app/#097f77d209302d5774f02ea42db81b460a2f149f3ab525a40c5b26c8ab41d843 Chat recovered https://status.prod.unique.app/ Tue, 16 Dec 2025 10:39:18 +0000 https://status.prod.unique.app/#5e5e45885cdfa370acfd7b834c2cb0b4716f7159f803317e19a3e21ad2220f13 Sidebar recovered https://status.prod.unique.app/ Tue, 16 Dec 2025 10:39:07 +0000 https://status.prod.unique.app/#aebc2d8eb5fcff76995cd9d4e2349b98db2af960f8132c95ba9a2158f8f0c106 Sidebar recovered https://status.prod.unique.app/ Tue, 16 Dec 2025 10:39:00 +0000 https://status.prod.unique.app/#8015cad96c32b90df80a41e21a206cd6fd273d942f5b77a8359376af975b9695 IDP recovered https://status.prod.unique.app/ Tue, 16 Dec 2025 10:38:27 +0000 https://status.prod.unique.app/#e680c63bdab4c66c4886dc6503a0c3a94160683a16ee8c8e4e74481cd77b8e21 Chat recovered https://status.prod.unique.app/ Tue, 16 Dec 2025 10:38:16 +0000 https://status.prod.unique.app/#be146d5a5f0a11520af3bde865e782e2bf6308d221a8913e209f363d63217bba Microsoft recovered https://status.prod.unique.app/ Tue, 16 Dec 2025 10:38:06 +0000 https://status.prod.unique.app/#e29ecfda6864f8cc03435fcfdc8fbdbe4dbf96e6238e5ee020d43e2035d23313 Scope Management recovered https://status.prod.unique.app/ Tue, 16 Dec 2025 10:38:04 +0000 https://status.prod.unique.app/#ebfb6d97c4516adf035e56cd5af146e7ae62ba4f429cacdbb62a8efca1c1a797 Recording recovered https://status.prod.unique.app/ Tue, 16 Dec 2025 10:37:55 +0000 https://status.prod.unique.app/#11fb1a5a7abcc354873dd598434ab36f8f5b60d918d5847d8a6279f62ef241a8 Webhooks recovered https://status.prod.unique.app/ Tue, 16 Dec 2025 10:35:22 +0000 https://status.prod.unique.app/#e680c63bdab4c66c4886dc6503a0c3a94160683a16ee8c8e4e74481cd77b8e21 Chat went down https://status.prod.unique.app/ Tue, 16 Dec 2025 10:35:17 +0000 https://status.prod.unique.app/#be146d5a5f0a11520af3bde865e782e2bf6308d221a8913e209f363d63217bba Microsoft went down https://status.prod.unique.app/ Tue, 16 Dec 2025 10:35:10 +0000 https://status.prod.unique.app/#e29ecfda6864f8cc03435fcfdc8fbdbe4dbf96e6238e5ee020d43e2035d23313 Scope Management went down https://status.prod.unique.app/ Tue, 16 Dec 2025 10:35:06 +0000 https://status.prod.unique.app/#ebfb6d97c4516adf035e56cd5af146e7ae62ba4f429cacdbb62a8efca1c1a797 Recording went down https://status.prod.unique.app/ Tue, 16 Dec 2025 10:35:00 +0000 https://status.prod.unique.app/#11fb1a5a7abcc354873dd598434ab36f8f5b60d918d5847d8a6279f62ef241a8 Webhooks went down https://status.prod.unique.app/ Tue, 16 Dec 2025 10:34:03 +0000 https://status.prod.unique.app/#ac618714db0b40ddfe37f198940d48c1a762200ec9f1c0e952d3c22225e98f06 Knowledge Base went down https://status.prod.unique.app/ Tue, 16 Dec 2025 10:33:42 +0000 https://status.prod.unique.app/#23b02dcb33fc37660e7e7999d2b595fe4c416adeb2286be9887a57b03e70f392 Recording went down https://status.prod.unique.app/ Tue, 16 Dec 2025 10:33:35 +0000 https://status.prod.unique.app/#097f77d209302d5774f02ea42db81b460a2f149f3ab525a40c5b26c8ab41d843 Chat went down https://status.prod.unique.app/ Tue, 16 Dec 2025 10:33:16 +0000 https://status.prod.unique.app/#5e5e45885cdfa370acfd7b834c2cb0b4716f7159f803317e19a3e21ad2220f13 Sidebar went down https://status.prod.unique.app/ Tue, 16 Dec 2025 10:33:07 +0000 https://status.prod.unique.app/#aebc2d8eb5fcff76995cd9d4e2349b98db2af960f8132c95ba9a2158f8f0c106 Sidebar went down https://status.prod.unique.app/ Tue, 16 Dec 2025 10:32:56 +0000 https://status.prod.unique.app/#8015cad96c32b90df80a41e21a206cd6fd273d942f5b77a8359376af975b9695 IDP went down https://status.prod.unique.app/ Thu, 04 Dec 2025 23:49:06 +0000 https://status.prod.unique.app/#a0076f74bbe4e26dd8c6a0336d1a96cd868983b991b41c3ff5768049932ea171 Sidebar recovered https://status.prod.unique.app/ Thu, 04 Dec 2025 23:45:47 +0000 https://status.prod.unique.app/#9a958c82a9a7e65aa24f64b488be7be482842be117b25c19dd199f4c04af4244 IDP recovered https://status.prod.unique.app/ Thu, 04 Dec 2025 23:31:07 +0000 https://status.prod.unique.app/#a0076f74bbe4e26dd8c6a0336d1a96cd868983b991b41c3ff5768049932ea171 Sidebar went down https://status.prod.unique.app/ Thu, 04 Dec 2025 23:21:45 +0000 https://status.prod.unique.app/#9a958c82a9a7e65aa24f64b488be7be482842be117b25c19dd199f4c04af4244 IDP went down https://status.prod.unique.app/ Wed, 03 Dec 2025 17:36:35 +0000 https://status.prod.unique.app/#cc0f4d4c65447066c813a488b5b822a77af94f01b6fa816436771fa135767f17 Sidebar recovered https://status.prod.unique.app/ Wed, 03 Dec 2025 17:33:33 +0000 https://status.prod.unique.app/#cc0f4d4c65447066c813a488b5b822a77af94f01b6fa816436771fa135767f17 Sidebar went down https://status.prod.unique.app/ Wed, 03 Dec 2025 08:44:47 +0000 https://status.prod.unique.app/#02a51bff9a3f3426293efe88e6e0a258f327e7a66f2c5df2a678f3dd1aff135d Sidebar recovered https://status.prod.unique.app/ Wed, 03 Dec 2025 08:41:46 +0000 https://status.prod.unique.app/#02a51bff9a3f3426293efe88e6e0a258f327e7a66f2c5df2a678f3dd1aff135d Sidebar went down https://status.prod.unique.app/ Tue, 02 Dec 2025 14:53:22 +0000 https://status.prod.unique.app/#992f6c1a34203e5c8c3b5157316a01f19ae9b053190af6a305a253d746d43158 Sidebar recovered https://status.prod.unique.app/ Tue, 02 Dec 2025 14:50:21 +0000 https://status.prod.unique.app/#992f6c1a34203e5c8c3b5157316a01f19ae9b053190af6a305a253d746d43158 Sidebar went down https://status.prod.unique.app/ Thu, 27 Nov 2025 17:23:05 +0000 https://status.prod.unique.app/#8688d6a7bfda489a38a31acf92a60938fa67b41e057c3351f039f4e8aedc5fc8 Webhooks recovered https://status.prod.unique.app/ Thu, 27 Nov 2025 17:21:25 +0000 https://status.prod.unique.app/#6d5bdc1703755443db79c343521b38a06f5b68b8ee197ab921f52024534cf228 Sidebar recovered https://status.prod.unique.app/ Thu, 27 Nov 2025 17:20:04 +0000 https://status.prod.unique.app/#8688d6a7bfda489a38a31acf92a60938fa67b41e057c3351f039f4e8aedc5fc8 Webhooks went down https://status.prod.unique.app/ Thu, 27 Nov 2025 17:18:22 +0000 https://status.prod.unique.app/#6d5bdc1703755443db79c343521b38a06f5b68b8ee197ab921f52024534cf228 Sidebar went down https://status.prod.unique.app/ Tue, 25 Nov 2025 17:30:45 +0000 https://status.prod.unique.app/#439c62d6690191e1f4fbd389132e1f83fd39533ee93c4772b3354c787ca9d547 Sidebar recovered https://status.prod.unique.app/ Tue, 25 Nov 2025 17:27:46 +0000 https://status.prod.unique.app/#439c62d6690191e1f4fbd389132e1f83fd39533ee93c4772b3354c787ca9d547 Sidebar went down https://status.prod.unique.app/ Mon, 24 Nov 2025 11:14:24 +0000 https://status.prod.unique.app/#a4cbce4702e0322d19b73ce74bccee657946bbc2c0f2d79e0ff6e07096312440 Sidebar recovered https://status.prod.unique.app/ Mon, 24 Nov 2025 11:11:22 +0000 https://status.prod.unique.app/#a4cbce4702e0322d19b73ce74bccee657946bbc2c0f2d79e0ff6e07096312440 Sidebar went down https://status.prod.unique.app/incident/546339 Wed, 16 Apr 2025 19:57:25 -0000 https://status.prod.unique.app/incident/546339#3b9fe53beb85191ced92eca7c428b64da77d970dcc50b22ce248e122fa604a8e Maintenance completed https://status.prod.unique.app/incident/546339 Wed, 16 Apr 2025 19:57:25 -0000 https://status.prod.unique.app/incident/546339#3b9fe53beb85191ced92eca7c428b64da77d970dcc50b22ce248e122fa604a8e Maintenance completed https://status.prod.unique.app/incident/546339 Wed, 16 Apr 2025 19:57:25 -0000 https://status.prod.unique.app/incident/546339#3b9fe53beb85191ced92eca7c428b64da77d970dcc50b22ce248e122fa604a8e Maintenance completed https://status.prod.unique.app/incident/546339 Wed, 16 Apr 2025 19:57:25 -0000 https://status.prod.unique.app/incident/546339#3b9fe53beb85191ced92eca7c428b64da77d970dcc50b22ce248e122fa604a8e Maintenance completed https://status.prod.unique.app/incident/546339 Wed, 16 Apr 2025 19:57:25 -0000 https://status.prod.unique.app/incident/546339#3b9fe53beb85191ced92eca7c428b64da77d970dcc50b22ce248e122fa604a8e Maintenance completed https://status.prod.unique.app/incident/546339 Wed, 16 Apr 2025 19:57:25 -0000 https://status.prod.unique.app/incident/546339#3b9fe53beb85191ced92eca7c428b64da77d970dcc50b22ce248e122fa604a8e Maintenance completed https://status.prod.unique.app/incident/546339 Wed, 16 Apr 2025 19:57:25 -0000 https://status.prod.unique.app/incident/546339#3b9fe53beb85191ced92eca7c428b64da77d970dcc50b22ce248e122fa604a8e Maintenance completed https://status.prod.unique.app/incident/546339 Wed, 16 Apr 2025 19:57:25 -0000 https://status.prod.unique.app/incident/546339#3b9fe53beb85191ced92eca7c428b64da77d970dcc50b22ce248e122fa604a8e Maintenance completed https://status.prod.unique.app/incident/546339 Wed, 16 Apr 2025 19:57:25 -0000 https://status.prod.unique.app/incident/546339#3b9fe53beb85191ced92eca7c428b64da77d970dcc50b22ce248e122fa604a8e Maintenance completed https://status.prod.unique.app/incident/546339 Wed, 16 Apr 2025 19:57:25 -0000 https://status.prod.unique.app/incident/546339#3b9fe53beb85191ced92eca7c428b64da77d970dcc50b22ce248e122fa604a8e Maintenance completed https://status.prod.unique.app/incident/546339 Wed, 16 Apr 2025 19:57:25 -0000 https://status.prod.unique.app/incident/546339#3b9fe53beb85191ced92eca7c428b64da77d970dcc50b22ce248e122fa604a8e Maintenance completed https://status.prod.unique.app/incident/546339 Wed, 16 Apr 2025 19:57:25 -0000 https://status.prod.unique.app/incident/546339#3b9fe53beb85191ced92eca7c428b64da77d970dcc50b22ce248e122fa604a8e Maintenance completed https://status.prod.unique.app/incident/546339 Wed, 16 Apr 2025 19:00:58 -0000 https://status.prod.unique.app/incident/546339#ee60abf751b9553a204cd7c386b1110113a65329cd1ffe70b353b6404d00ac93 We upgraded the underlying Kubernetes cluster to a newer version. No noticeable degradations occurred. https://status.prod.unique.app/incident/546339 Wed, 16 Apr 2025 19:00:58 -0000 https://status.prod.unique.app/incident/546339#ee60abf751b9553a204cd7c386b1110113a65329cd1ffe70b353b6404d00ac93 We upgraded the underlying Kubernetes cluster to a newer version. No noticeable degradations occurred. https://status.prod.unique.app/incident/546339 Wed, 16 Apr 2025 19:00:58 -0000 https://status.prod.unique.app/incident/546339#ee60abf751b9553a204cd7c386b1110113a65329cd1ffe70b353b6404d00ac93 We upgraded the underlying Kubernetes cluster to a newer version. No noticeable degradations occurred. https://status.prod.unique.app/incident/546339 Wed, 16 Apr 2025 19:00:58 -0000 https://status.prod.unique.app/incident/546339#ee60abf751b9553a204cd7c386b1110113a65329cd1ffe70b353b6404d00ac93 We upgraded the underlying Kubernetes cluster to a newer version. No noticeable degradations occurred. https://status.prod.unique.app/incident/546339 Wed, 16 Apr 2025 19:00:58 -0000 https://status.prod.unique.app/incident/546339#ee60abf751b9553a204cd7c386b1110113a65329cd1ffe70b353b6404d00ac93 We upgraded the underlying Kubernetes cluster to a newer version. No noticeable degradations occurred. https://status.prod.unique.app/incident/546339 Wed, 16 Apr 2025 19:00:58 -0000 https://status.prod.unique.app/incident/546339#ee60abf751b9553a204cd7c386b1110113a65329cd1ffe70b353b6404d00ac93 We upgraded the underlying Kubernetes cluster to a newer version. No noticeable degradations occurred. https://status.prod.unique.app/incident/546339 Wed, 16 Apr 2025 19:00:58 -0000 https://status.prod.unique.app/incident/546339#ee60abf751b9553a204cd7c386b1110113a65329cd1ffe70b353b6404d00ac93 We upgraded the underlying Kubernetes cluster to a newer version. No noticeable degradations occurred. https://status.prod.unique.app/incident/546339 Wed, 16 Apr 2025 19:00:58 -0000 https://status.prod.unique.app/incident/546339#ee60abf751b9553a204cd7c386b1110113a65329cd1ffe70b353b6404d00ac93 We upgraded the underlying Kubernetes cluster to a newer version. No noticeable degradations occurred. https://status.prod.unique.app/incident/546339 Wed, 16 Apr 2025 19:00:58 -0000 https://status.prod.unique.app/incident/546339#ee60abf751b9553a204cd7c386b1110113a65329cd1ffe70b353b6404d00ac93 We upgraded the underlying Kubernetes cluster to a newer version. No noticeable degradations occurred. https://status.prod.unique.app/incident/546339 Wed, 16 Apr 2025 19:00:58 -0000 https://status.prod.unique.app/incident/546339#ee60abf751b9553a204cd7c386b1110113a65329cd1ffe70b353b6404d00ac93 We upgraded the underlying Kubernetes cluster to a newer version. No noticeable degradations occurred. https://status.prod.unique.app/incident/546339 Wed, 16 Apr 2025 19:00:58 -0000 https://status.prod.unique.app/incident/546339#ee60abf751b9553a204cd7c386b1110113a65329cd1ffe70b353b6404d00ac93 We upgraded the underlying Kubernetes cluster to a newer version. No noticeable degradations occurred. https://status.prod.unique.app/incident/546339 Wed, 16 Apr 2025 19:00:58 -0000 https://status.prod.unique.app/incident/546339#ee60abf751b9553a204cd7c386b1110113a65329cd1ffe70b353b6404d00ac93 We upgraded the underlying Kubernetes cluster to a newer version. No noticeable degradations occurred. https://status.prod.unique.app/incident/515093 Wed, 19 Feb 2025 00:00:00 -0000 https://status.prod.unique.app/incident/515093#fdaca75094cf09d4ade66094e8ee99e82ee9681b655b4935b1f448251fde8169 Maintenance completed https://status.prod.unique.app/incident/515093 Wed, 19 Feb 2025 00:00:00 -0000 https://status.prod.unique.app/incident/515093#fdaca75094cf09d4ade66094e8ee99e82ee9681b655b4935b1f448251fde8169 Maintenance completed https://status.prod.unique.app/incident/515093 Wed, 19 Feb 2025 00:00:00 -0000 https://status.prod.unique.app/incident/515093#fdaca75094cf09d4ade66094e8ee99e82ee9681b655b4935b1f448251fde8169 Maintenance completed https://status.prod.unique.app/incident/515093 Wed, 19 Feb 2025 00:00:00 -0000 https://status.prod.unique.app/incident/515093#fdaca75094cf09d4ade66094e8ee99e82ee9681b655b4935b1f448251fde8169 Maintenance completed https://status.prod.unique.app/incident/515093 Wed, 19 Feb 2025 00:00:00 -0000 https://status.prod.unique.app/incident/515093#fdaca75094cf09d4ade66094e8ee99e82ee9681b655b4935b1f448251fde8169 Maintenance completed https://status.prod.unique.app/incident/515093 Wed, 19 Feb 2025 00:00:00 -0000 https://status.prod.unique.app/incident/515093#fdaca75094cf09d4ade66094e8ee99e82ee9681b655b4935b1f448251fde8169 Maintenance completed https://status.prod.unique.app/incident/515093 Wed, 19 Feb 2025 00:00:00 -0000 https://status.prod.unique.app/incident/515093#fdaca75094cf09d4ade66094e8ee99e82ee9681b655b4935b1f448251fde8169 Maintenance completed https://status.prod.unique.app/incident/515093 Wed, 19 Feb 2025 00:00:00 -0000 https://status.prod.unique.app/incident/515093#fdaca75094cf09d4ade66094e8ee99e82ee9681b655b4935b1f448251fde8169 Maintenance completed https://status.prod.unique.app/incident/515093 Wed, 19 Feb 2025 00:00:00 -0000 https://status.prod.unique.app/incident/515093#fdaca75094cf09d4ade66094e8ee99e82ee9681b655b4935b1f448251fde8169 Maintenance completed https://status.prod.unique.app/incident/515093 Wed, 19 Feb 2025 00:00:00 -0000 https://status.prod.unique.app/incident/515093#fdaca75094cf09d4ade66094e8ee99e82ee9681b655b4935b1f448251fde8169 Maintenance completed https://status.prod.unique.app/incident/515093 Wed, 19 Feb 2025 00:00:00 -0000 https://status.prod.unique.app/incident/515093#fdaca75094cf09d4ade66094e8ee99e82ee9681b655b4935b1f448251fde8169 Maintenance completed https://status.prod.unique.app/incident/515093 Wed, 19 Feb 2025 00:00:00 -0000 https://status.prod.unique.app/incident/515093#fdaca75094cf09d4ade66094e8ee99e82ee9681b655b4935b1f448251fde8169 Maintenance completed https://status.prod.unique.app/incident/515093 Tue, 18 Feb 2025 21:00:00 -0000 https://status.prod.unique.app/incident/515093#2460a8166702ff6e7e035fc828c55611311cfe2c1c3ef40681c090fe1475a830 Correcting https://status.prod.unique.app/incident/514786, cluster will experience downtime https://status.prod.unique.app/incident/515093 Tue, 18 Feb 2025 21:00:00 -0000 https://status.prod.unique.app/incident/515093#2460a8166702ff6e7e035fc828c55611311cfe2c1c3ef40681c090fe1475a830 Correcting https://status.prod.unique.app/incident/514786, cluster will experience downtime https://status.prod.unique.app/incident/515093 Tue, 18 Feb 2025 21:00:00 -0000 https://status.prod.unique.app/incident/515093#2460a8166702ff6e7e035fc828c55611311cfe2c1c3ef40681c090fe1475a830 Correcting https://status.prod.unique.app/incident/514786, cluster will experience downtime https://status.prod.unique.app/incident/515093 Tue, 18 Feb 2025 21:00:00 -0000 https://status.prod.unique.app/incident/515093#2460a8166702ff6e7e035fc828c55611311cfe2c1c3ef40681c090fe1475a830 Correcting https://status.prod.unique.app/incident/514786, cluster will experience downtime https://status.prod.unique.app/incident/515093 Tue, 18 Feb 2025 21:00:00 -0000 https://status.prod.unique.app/incident/515093#2460a8166702ff6e7e035fc828c55611311cfe2c1c3ef40681c090fe1475a830 Correcting https://status.prod.unique.app/incident/514786, cluster will experience downtime https://status.prod.unique.app/incident/515093 Tue, 18 Feb 2025 21:00:00 -0000 https://status.prod.unique.app/incident/515093#2460a8166702ff6e7e035fc828c55611311cfe2c1c3ef40681c090fe1475a830 Correcting https://status.prod.unique.app/incident/514786, cluster will experience downtime https://status.prod.unique.app/incident/515093 Tue, 18 Feb 2025 21:00:00 -0000 https://status.prod.unique.app/incident/515093#2460a8166702ff6e7e035fc828c55611311cfe2c1c3ef40681c090fe1475a830 Correcting https://status.prod.unique.app/incident/514786, cluster will experience downtime https://status.prod.unique.app/incident/515093 Tue, 18 Feb 2025 21:00:00 -0000 https://status.prod.unique.app/incident/515093#2460a8166702ff6e7e035fc828c55611311cfe2c1c3ef40681c090fe1475a830 Correcting https://status.prod.unique.app/incident/514786, cluster will experience downtime https://status.prod.unique.app/incident/515093 Tue, 18 Feb 2025 21:00:00 -0000 https://status.prod.unique.app/incident/515093#2460a8166702ff6e7e035fc828c55611311cfe2c1c3ef40681c090fe1475a830 Correcting https://status.prod.unique.app/incident/514786, cluster will experience downtime https://status.prod.unique.app/incident/515093 Tue, 18 Feb 2025 21:00:00 -0000 https://status.prod.unique.app/incident/515093#2460a8166702ff6e7e035fc828c55611311cfe2c1c3ef40681c090fe1475a830 Correcting https://status.prod.unique.app/incident/514786, cluster will experience downtime https://status.prod.unique.app/incident/515093 Tue, 18 Feb 2025 21:00:00 -0000 https://status.prod.unique.app/incident/515093#2460a8166702ff6e7e035fc828c55611311cfe2c1c3ef40681c090fe1475a830 Correcting https://status.prod.unique.app/incident/514786, cluster will experience downtime https://status.prod.unique.app/incident/515093 Tue, 18 Feb 2025 21:00:00 -0000 https://status.prod.unique.app/incident/515093#2460a8166702ff6e7e035fc828c55611311cfe2c1c3ef40681c090fe1475a830 Correcting https://status.prod.unique.app/incident/514786, cluster will experience downtime https://status.prod.unique.app/incident/514786 Mon, 17 Feb 2025 16:47:00 -0000 https://status.prod.unique.app/incident/514786#16ea211542e44dd3ae16e9605d5b5af1132827df5161842ec44b75e2ffadd73a # Postmortem: Involuntary Security Upgrade ## Incident Timeline 1. **14:43 CET**: Initiation of a security upgrade. 2. **14:58 CET**: First service began experiencing significant degradations, escalating to disruptions. 3. **14:58 - 16:34 CET**: Continued degradations and disruptions as the upgrade progressed. 4. **16:34 CET**: Completion of the upgrade, restoring the cluster to its operational state. 5. **Post-Upgrade Inspection**: Document ingestion was found to be non-functional. 6. **17:47 CET**: Document ingestion pipelines were repaired, fully resolving the incident. ## Root Cause Analysis 1. The upgrade was intended to update the system to a newer Kubernetes version to enhance maintainability and security. 2. The upgrade should have been scheduled outside of business hours to minimize impact during periods of high load. 3. An uncoordinated change led to the upgrade being triggered prematurely during peak operational hours. 4. This premature upgrade caused significant strain on Kubernetes, leading to the observed service disruptions. ## Impact Assessment - **Duration of Severe Degradations**: 1 hour and 33 minutes (partial system availability was maintained). - **Affected Services**: unique.app multitenant features (Chat and Recording). Other tenants remained unaffected. ## Preventative Measures 1. Implement additional safeguards in the infrastructure upgrade process to prevent unscheduled changes. 2. Allocate more resources during daytime operations to allow for cluster rebalancing, reducing the impact of involuntary disruptions. 3. Further segment unique workloads into smaller, independently upgradable pools to minimize the blast radius of similar incidents in the future. https://status.prod.unique.app/incident/514786 Mon, 17 Feb 2025 16:47:00 -0000 https://status.prod.unique.app/incident/514786#16ea211542e44dd3ae16e9605d5b5af1132827df5161842ec44b75e2ffadd73a # Postmortem: Involuntary Security Upgrade ## Incident Timeline 1. **14:43 CET**: Initiation of a security upgrade. 2. **14:58 CET**: First service began experiencing significant degradations, escalating to disruptions. 3. **14:58 - 16:34 CET**: Continued degradations and disruptions as the upgrade progressed. 4. **16:34 CET**: Completion of the upgrade, restoring the cluster to its operational state. 5. **Post-Upgrade Inspection**: Document ingestion was found to be non-functional. 6. **17:47 CET**: Document ingestion pipelines were repaired, fully resolving the incident. ## Root Cause Analysis 1. The upgrade was intended to update the system to a newer Kubernetes version to enhance maintainability and security. 2. The upgrade should have been scheduled outside of business hours to minimize impact during periods of high load. 3. An uncoordinated change led to the upgrade being triggered prematurely during peak operational hours. 4. This premature upgrade caused significant strain on Kubernetes, leading to the observed service disruptions. ## Impact Assessment - **Duration of Severe Degradations**: 1 hour and 33 minutes (partial system availability was maintained). - **Affected Services**: unique.app multitenant features (Chat and Recording). Other tenants remained unaffected. ## Preventative Measures 1. Implement additional safeguards in the infrastructure upgrade process to prevent unscheduled changes. 2. Allocate more resources during daytime operations to allow for cluster rebalancing, reducing the impact of involuntary disruptions. 3. Further segment unique workloads into smaller, independently upgradable pools to minimize the blast radius of similar incidents in the future. https://status.prod.unique.app/incident/514786 Mon, 17 Feb 2025 16:47:00 -0000 https://status.prod.unique.app/incident/514786#16ea211542e44dd3ae16e9605d5b5af1132827df5161842ec44b75e2ffadd73a # Postmortem: Involuntary Security Upgrade ## Incident Timeline 1. **14:43 CET**: Initiation of a security upgrade. 2. **14:58 CET**: First service began experiencing significant degradations, escalating to disruptions. 3. **14:58 - 16:34 CET**: Continued degradations and disruptions as the upgrade progressed. 4. **16:34 CET**: Completion of the upgrade, restoring the cluster to its operational state. 5. **Post-Upgrade Inspection**: Document ingestion was found to be non-functional. 6. **17:47 CET**: Document ingestion pipelines were repaired, fully resolving the incident. ## Root Cause Analysis 1. The upgrade was intended to update the system to a newer Kubernetes version to enhance maintainability and security. 2. The upgrade should have been scheduled outside of business hours to minimize impact during periods of high load. 3. An uncoordinated change led to the upgrade being triggered prematurely during peak operational hours. 4. This premature upgrade caused significant strain on Kubernetes, leading to the observed service disruptions. ## Impact Assessment - **Duration of Severe Degradations**: 1 hour and 33 minutes (partial system availability was maintained). - **Affected Services**: unique.app multitenant features (Chat and Recording). Other tenants remained unaffected. ## Preventative Measures 1. Implement additional safeguards in the infrastructure upgrade process to prevent unscheduled changes. 2. Allocate more resources during daytime operations to allow for cluster rebalancing, reducing the impact of involuntary disruptions. 3. Further segment unique workloads into smaller, independently upgradable pools to minimize the blast radius of similar incidents in the future. https://status.prod.unique.app/incident/514786 Mon, 17 Feb 2025 16:47:00 -0000 https://status.prod.unique.app/incident/514786#16ea211542e44dd3ae16e9605d5b5af1132827df5161842ec44b75e2ffadd73a # Postmortem: Involuntary Security Upgrade ## Incident Timeline 1. **14:43 CET**: Initiation of a security upgrade. 2. **14:58 CET**: First service began experiencing significant degradations, escalating to disruptions. 3. **14:58 - 16:34 CET**: Continued degradations and disruptions as the upgrade progressed. 4. **16:34 CET**: Completion of the upgrade, restoring the cluster to its operational state. 5. **Post-Upgrade Inspection**: Document ingestion was found to be non-functional. 6. **17:47 CET**: Document ingestion pipelines were repaired, fully resolving the incident. ## Root Cause Analysis 1. The upgrade was intended to update the system to a newer Kubernetes version to enhance maintainability and security. 2. The upgrade should have been scheduled outside of business hours to minimize impact during periods of high load. 3. An uncoordinated change led to the upgrade being triggered prematurely during peak operational hours. 4. This premature upgrade caused significant strain on Kubernetes, leading to the observed service disruptions. ## Impact Assessment - **Duration of Severe Degradations**: 1 hour and 33 minutes (partial system availability was maintained). - **Affected Services**: unique.app multitenant features (Chat and Recording). Other tenants remained unaffected. ## Preventative Measures 1. Implement additional safeguards in the infrastructure upgrade process to prevent unscheduled changes. 2. Allocate more resources during daytime operations to allow for cluster rebalancing, reducing the impact of involuntary disruptions. 3. Further segment unique workloads into smaller, independently upgradable pools to minimize the blast radius of similar incidents in the future. https://status.prod.unique.app/incident/514786 Mon, 17 Feb 2025 16:47:00 -0000 https://status.prod.unique.app/incident/514786#16ea211542e44dd3ae16e9605d5b5af1132827df5161842ec44b75e2ffadd73a # Postmortem: Involuntary Security Upgrade ## Incident Timeline 1. **14:43 CET**: Initiation of a security upgrade. 2. **14:58 CET**: First service began experiencing significant degradations, escalating to disruptions. 3. **14:58 - 16:34 CET**: Continued degradations and disruptions as the upgrade progressed. 4. **16:34 CET**: Completion of the upgrade, restoring the cluster to its operational state. 5. **Post-Upgrade Inspection**: Document ingestion was found to be non-functional. 6. **17:47 CET**: Document ingestion pipelines were repaired, fully resolving the incident. ## Root Cause Analysis 1. The upgrade was intended to update the system to a newer Kubernetes version to enhance maintainability and security. 2. The upgrade should have been scheduled outside of business hours to minimize impact during periods of high load. 3. An uncoordinated change led to the upgrade being triggered prematurely during peak operational hours. 4. This premature upgrade caused significant strain on Kubernetes, leading to the observed service disruptions. ## Impact Assessment - **Duration of Severe Degradations**: 1 hour and 33 minutes (partial system availability was maintained). - **Affected Services**: unique.app multitenant features (Chat and Recording). Other tenants remained unaffected. ## Preventative Measures 1. Implement additional safeguards in the infrastructure upgrade process to prevent unscheduled changes. 2. Allocate more resources during daytime operations to allow for cluster rebalancing, reducing the impact of involuntary disruptions. 3. Further segment unique workloads into smaller, independently upgradable pools to minimize the blast radius of similar incidents in the future. https://status.prod.unique.app/incident/514786 Mon, 17 Feb 2025 16:47:00 -0000 https://status.prod.unique.app/incident/514786#16ea211542e44dd3ae16e9605d5b5af1132827df5161842ec44b75e2ffadd73a # Postmortem: Involuntary Security Upgrade ## Incident Timeline 1. **14:43 CET**: Initiation of a security upgrade. 2. **14:58 CET**: First service began experiencing significant degradations, escalating to disruptions. 3. **14:58 - 16:34 CET**: Continued degradations and disruptions as the upgrade progressed. 4. **16:34 CET**: Completion of the upgrade, restoring the cluster to its operational state. 5. **Post-Upgrade Inspection**: Document ingestion was found to be non-functional. 6. **17:47 CET**: Document ingestion pipelines were repaired, fully resolving the incident. ## Root Cause Analysis 1. The upgrade was intended to update the system to a newer Kubernetes version to enhance maintainability and security. 2. The upgrade should have been scheduled outside of business hours to minimize impact during periods of high load. 3. An uncoordinated change led to the upgrade being triggered prematurely during peak operational hours. 4. This premature upgrade caused significant strain on Kubernetes, leading to the observed service disruptions. ## Impact Assessment - **Duration of Severe Degradations**: 1 hour and 33 minutes (partial system availability was maintained). - **Affected Services**: unique.app multitenant features (Chat and Recording). Other tenants remained unaffected. ## Preventative Measures 1. Implement additional safeguards in the infrastructure upgrade process to prevent unscheduled changes. 2. Allocate more resources during daytime operations to allow for cluster rebalancing, reducing the impact of involuntary disruptions. 3. Further segment unique workloads into smaller, independently upgradable pools to minimize the blast radius of similar incidents in the future. https://status.prod.unique.app/incident/514786 Mon, 17 Feb 2025 16:47:00 -0000 https://status.prod.unique.app/incident/514786#16ea211542e44dd3ae16e9605d5b5af1132827df5161842ec44b75e2ffadd73a # Postmortem: Involuntary Security Upgrade ## Incident Timeline 1. **14:43 CET**: Initiation of a security upgrade. 2. **14:58 CET**: First service began experiencing significant degradations, escalating to disruptions. 3. **14:58 - 16:34 CET**: Continued degradations and disruptions as the upgrade progressed. 4. **16:34 CET**: Completion of the upgrade, restoring the cluster to its operational state. 5. **Post-Upgrade Inspection**: Document ingestion was found to be non-functional. 6. **17:47 CET**: Document ingestion pipelines were repaired, fully resolving the incident. ## Root Cause Analysis 1. The upgrade was intended to update the system to a newer Kubernetes version to enhance maintainability and security. 2. The upgrade should have been scheduled outside of business hours to minimize impact during periods of high load. 3. An uncoordinated change led to the upgrade being triggered prematurely during peak operational hours. 4. This premature upgrade caused significant strain on Kubernetes, leading to the observed service disruptions. ## Impact Assessment - **Duration of Severe Degradations**: 1 hour and 33 minutes (partial system availability was maintained). - **Affected Services**: unique.app multitenant features (Chat and Recording). Other tenants remained unaffected. ## Preventative Measures 1. Implement additional safeguards in the infrastructure upgrade process to prevent unscheduled changes. 2. Allocate more resources during daytime operations to allow for cluster rebalancing, reducing the impact of involuntary disruptions. 3. Further segment unique workloads into smaller, independently upgradable pools to minimize the blast radius of similar incidents in the future. https://status.prod.unique.app/incident/514786 Mon, 17 Feb 2025 16:47:00 -0000 https://status.prod.unique.app/incident/514786#16ea211542e44dd3ae16e9605d5b5af1132827df5161842ec44b75e2ffadd73a # Postmortem: Involuntary Security Upgrade ## Incident Timeline 1. **14:43 CET**: Initiation of a security upgrade. 2. **14:58 CET**: First service began experiencing significant degradations, escalating to disruptions. 3. **14:58 - 16:34 CET**: Continued degradations and disruptions as the upgrade progressed. 4. **16:34 CET**: Completion of the upgrade, restoring the cluster to its operational state. 5. **Post-Upgrade Inspection**: Document ingestion was found to be non-functional. 6. **17:47 CET**: Document ingestion pipelines were repaired, fully resolving the incident. ## Root Cause Analysis 1. The upgrade was intended to update the system to a newer Kubernetes version to enhance maintainability and security. 2. The upgrade should have been scheduled outside of business hours to minimize impact during periods of high load. 3. An uncoordinated change led to the upgrade being triggered prematurely during peak operational hours. 4. This premature upgrade caused significant strain on Kubernetes, leading to the observed service disruptions. ## Impact Assessment - **Duration of Severe Degradations**: 1 hour and 33 minutes (partial system availability was maintained). - **Affected Services**: unique.app multitenant features (Chat and Recording). Other tenants remained unaffected. ## Preventative Measures 1. Implement additional safeguards in the infrastructure upgrade process to prevent unscheduled changes. 2. Allocate more resources during daytime operations to allow for cluster rebalancing, reducing the impact of involuntary disruptions. 3. Further segment unique workloads into smaller, independently upgradable pools to minimize the blast radius of similar incidents in the future. https://status.prod.unique.app/incident/514786 Mon, 17 Feb 2025 16:47:00 -0000 https://status.prod.unique.app/incident/514786#16ea211542e44dd3ae16e9605d5b5af1132827df5161842ec44b75e2ffadd73a # Postmortem: Involuntary Security Upgrade ## Incident Timeline 1. **14:43 CET**: Initiation of a security upgrade. 2. **14:58 CET**: First service began experiencing significant degradations, escalating to disruptions. 3. **14:58 - 16:34 CET**: Continued degradations and disruptions as the upgrade progressed. 4. **16:34 CET**: Completion of the upgrade, restoring the cluster to its operational state. 5. **Post-Upgrade Inspection**: Document ingestion was found to be non-functional. 6. **17:47 CET**: Document ingestion pipelines were repaired, fully resolving the incident. ## Root Cause Analysis 1. The upgrade was intended to update the system to a newer Kubernetes version to enhance maintainability and security. 2. The upgrade should have been scheduled outside of business hours to minimize impact during periods of high load. 3. An uncoordinated change led to the upgrade being triggered prematurely during peak operational hours. 4. This premature upgrade caused significant strain on Kubernetes, leading to the observed service disruptions. ## Impact Assessment - **Duration of Severe Degradations**: 1 hour and 33 minutes (partial system availability was maintained). - **Affected Services**: unique.app multitenant features (Chat and Recording). Other tenants remained unaffected. ## Preventative Measures 1. Implement additional safeguards in the infrastructure upgrade process to prevent unscheduled changes. 2. Allocate more resources during daytime operations to allow for cluster rebalancing, reducing the impact of involuntary disruptions. 3. Further segment unique workloads into smaller, independently upgradable pools to minimize the blast radius of similar incidents in the future. https://status.prod.unique.app/incident/514786 Mon, 17 Feb 2025 16:47:00 -0000 https://status.prod.unique.app/incident/514786#16ea211542e44dd3ae16e9605d5b5af1132827df5161842ec44b75e2ffadd73a # Postmortem: Involuntary Security Upgrade ## Incident Timeline 1. **14:43 CET**: Initiation of a security upgrade. 2. **14:58 CET**: First service began experiencing significant degradations, escalating to disruptions. 3. **14:58 - 16:34 CET**: Continued degradations and disruptions as the upgrade progressed. 4. **16:34 CET**: Completion of the upgrade, restoring the cluster to its operational state. 5. **Post-Upgrade Inspection**: Document ingestion was found to be non-functional. 6. **17:47 CET**: Document ingestion pipelines were repaired, fully resolving the incident. ## Root Cause Analysis 1. The upgrade was intended to update the system to a newer Kubernetes version to enhance maintainability and security. 2. The upgrade should have been scheduled outside of business hours to minimize impact during periods of high load. 3. An uncoordinated change led to the upgrade being triggered prematurely during peak operational hours. 4. This premature upgrade caused significant strain on Kubernetes, leading to the observed service disruptions. ## Impact Assessment - **Duration of Severe Degradations**: 1 hour and 33 minutes (partial system availability was maintained). - **Affected Services**: unique.app multitenant features (Chat and Recording). Other tenants remained unaffected. ## Preventative Measures 1. Implement additional safeguards in the infrastructure upgrade process to prevent unscheduled changes. 2. Allocate more resources during daytime operations to allow for cluster rebalancing, reducing the impact of involuntary disruptions. 3. Further segment unique workloads into smaller, independently upgradable pools to minimize the blast radius of similar incidents in the future. https://status.prod.unique.app/incident/514786 Mon, 17 Feb 2025 16:47:00 -0000 https://status.prod.unique.app/incident/514786#16ea211542e44dd3ae16e9605d5b5af1132827df5161842ec44b75e2ffadd73a # Postmortem: Involuntary Security Upgrade ## Incident Timeline 1. **14:43 CET**: Initiation of a security upgrade. 2. **14:58 CET**: First service began experiencing significant degradations, escalating to disruptions. 3. **14:58 - 16:34 CET**: Continued degradations and disruptions as the upgrade progressed. 4. **16:34 CET**: Completion of the upgrade, restoring the cluster to its operational state. 5. **Post-Upgrade Inspection**: Document ingestion was found to be non-functional. 6. **17:47 CET**: Document ingestion pipelines were repaired, fully resolving the incident. ## Root Cause Analysis 1. The upgrade was intended to update the system to a newer Kubernetes version to enhance maintainability and security. 2. The upgrade should have been scheduled outside of business hours to minimize impact during periods of high load. 3. An uncoordinated change led to the upgrade being triggered prematurely during peak operational hours. 4. This premature upgrade caused significant strain on Kubernetes, leading to the observed service disruptions. ## Impact Assessment - **Duration of Severe Degradations**: 1 hour and 33 minutes (partial system availability was maintained). - **Affected Services**: unique.app multitenant features (Chat and Recording). Other tenants remained unaffected. ## Preventative Measures 1. Implement additional safeguards in the infrastructure upgrade process to prevent unscheduled changes. 2. Allocate more resources during daytime operations to allow for cluster rebalancing, reducing the impact of involuntary disruptions. 3. Further segment unique workloads into smaller, independently upgradable pools to minimize the blast radius of similar incidents in the future. https://status.prod.unique.app/incident/514786 Mon, 17 Feb 2025 16:47:00 -0000 https://status.prod.unique.app/incident/514786#16ea211542e44dd3ae16e9605d5b5af1132827df5161842ec44b75e2ffadd73a # Postmortem: Involuntary Security Upgrade ## Incident Timeline 1. **14:43 CET**: Initiation of a security upgrade. 2. **14:58 CET**: First service began experiencing significant degradations, escalating to disruptions. 3. **14:58 - 16:34 CET**: Continued degradations and disruptions as the upgrade progressed. 4. **16:34 CET**: Completion of the upgrade, restoring the cluster to its operational state. 5. **Post-Upgrade Inspection**: Document ingestion was found to be non-functional. 6. **17:47 CET**: Document ingestion pipelines were repaired, fully resolving the incident. ## Root Cause Analysis 1. The upgrade was intended to update the system to a newer Kubernetes version to enhance maintainability and security. 2. The upgrade should have been scheduled outside of business hours to minimize impact during periods of high load. 3. An uncoordinated change led to the upgrade being triggered prematurely during peak operational hours. 4. This premature upgrade caused significant strain on Kubernetes, leading to the observed service disruptions. ## Impact Assessment - **Duration of Severe Degradations**: 1 hour and 33 minutes (partial system availability was maintained). - **Affected Services**: unique.app multitenant features (Chat and Recording). Other tenants remained unaffected. ## Preventative Measures 1. Implement additional safeguards in the infrastructure upgrade process to prevent unscheduled changes. 2. Allocate more resources during daytime operations to allow for cluster rebalancing, reducing the impact of involuntary disruptions. 3. Further segment unique workloads into smaller, independently upgradable pools to minimize the blast radius of similar incidents in the future. https://status.prod.unique.app/incident/514786 Mon, 17 Feb 2025 15:34:00 -0000 https://status.prod.unique.app/incident/514786#a75645934aa4b6c744560ef6d1e81b8e9f0cb16b987b6433edeabc01fb5592c8 An unplanned security upgrade to a newer Kubernetes version during peak hours caused significant service degradations and disruptions for 1 hour and 33 minutes, affecting Unique.app's multitenant features. The incident was resolved by 17:47 CET after repairing document ingestion pipelines. To prevent future occurrences, we will implement stricter safeguards, allocate more resources for daytime operations, and segment workloads for independent upgrades. https://status.prod.unique.app/incident/514786 Mon, 17 Feb 2025 15:34:00 -0000 https://status.prod.unique.app/incident/514786#a75645934aa4b6c744560ef6d1e81b8e9f0cb16b987b6433edeabc01fb5592c8 An unplanned security upgrade to a newer Kubernetes version during peak hours caused significant service degradations and disruptions for 1 hour and 33 minutes, affecting Unique.app's multitenant features. The incident was resolved by 17:47 CET after repairing document ingestion pipelines. To prevent future occurrences, we will implement stricter safeguards, allocate more resources for daytime operations, and segment workloads for independent upgrades. https://status.prod.unique.app/incident/514786 Mon, 17 Feb 2025 15:34:00 -0000 https://status.prod.unique.app/incident/514786#a75645934aa4b6c744560ef6d1e81b8e9f0cb16b987b6433edeabc01fb5592c8 An unplanned security upgrade to a newer Kubernetes version during peak hours caused significant service degradations and disruptions for 1 hour and 33 minutes, affecting Unique.app's multitenant features. The incident was resolved by 17:47 CET after repairing document ingestion pipelines. To prevent future occurrences, we will implement stricter safeguards, allocate more resources for daytime operations, and segment workloads for independent upgrades. https://status.prod.unique.app/incident/514786 Mon, 17 Feb 2025 15:34:00 -0000 https://status.prod.unique.app/incident/514786#a75645934aa4b6c744560ef6d1e81b8e9f0cb16b987b6433edeabc01fb5592c8 An unplanned security upgrade to a newer Kubernetes version during peak hours caused significant service degradations and disruptions for 1 hour and 33 minutes, affecting Unique.app's multitenant features. The incident was resolved by 17:47 CET after repairing document ingestion pipelines. To prevent future occurrences, we will implement stricter safeguards, allocate more resources for daytime operations, and segment workloads for independent upgrades. https://status.prod.unique.app/incident/514786 Mon, 17 Feb 2025 15:34:00 -0000 https://status.prod.unique.app/incident/514786#a75645934aa4b6c744560ef6d1e81b8e9f0cb16b987b6433edeabc01fb5592c8 An unplanned security upgrade to a newer Kubernetes version during peak hours caused significant service degradations and disruptions for 1 hour and 33 minutes, affecting Unique.app's multitenant features. The incident was resolved by 17:47 CET after repairing document ingestion pipelines. To prevent future occurrences, we will implement stricter safeguards, allocate more resources for daytime operations, and segment workloads for independent upgrades. https://status.prod.unique.app/incident/514786 Mon, 17 Feb 2025 15:34:00 -0000 https://status.prod.unique.app/incident/514786#a75645934aa4b6c744560ef6d1e81b8e9f0cb16b987b6433edeabc01fb5592c8 An unplanned security upgrade to a newer Kubernetes version during peak hours caused significant service degradations and disruptions for 1 hour and 33 minutes, affecting Unique.app's multitenant features. The incident was resolved by 17:47 CET after repairing document ingestion pipelines. To prevent future occurrences, we will implement stricter safeguards, allocate more resources for daytime operations, and segment workloads for independent upgrades. https://status.prod.unique.app/incident/514786 Mon, 17 Feb 2025 15:34:00 -0000 https://status.prod.unique.app/incident/514786#a75645934aa4b6c744560ef6d1e81b8e9f0cb16b987b6433edeabc01fb5592c8 An unplanned security upgrade to a newer Kubernetes version during peak hours caused significant service degradations and disruptions for 1 hour and 33 minutes, affecting Unique.app's multitenant features. The incident was resolved by 17:47 CET after repairing document ingestion pipelines. To prevent future occurrences, we will implement stricter safeguards, allocate more resources for daytime operations, and segment workloads for independent upgrades. https://status.prod.unique.app/incident/514786 Mon, 17 Feb 2025 15:34:00 -0000 https://status.prod.unique.app/incident/514786#a75645934aa4b6c744560ef6d1e81b8e9f0cb16b987b6433edeabc01fb5592c8 An unplanned security upgrade to a newer Kubernetes version during peak hours caused significant service degradations and disruptions for 1 hour and 33 minutes, affecting Unique.app's multitenant features. The incident was resolved by 17:47 CET after repairing document ingestion pipelines. To prevent future occurrences, we will implement stricter safeguards, allocate more resources for daytime operations, and segment workloads for independent upgrades. https://status.prod.unique.app/incident/514786 Mon, 17 Feb 2025 15:34:00 -0000 https://status.prod.unique.app/incident/514786#a75645934aa4b6c744560ef6d1e81b8e9f0cb16b987b6433edeabc01fb5592c8 An unplanned security upgrade to a newer Kubernetes version during peak hours caused significant service degradations and disruptions for 1 hour and 33 minutes, affecting Unique.app's multitenant features. The incident was resolved by 17:47 CET after repairing document ingestion pipelines. To prevent future occurrences, we will implement stricter safeguards, allocate more resources for daytime operations, and segment workloads for independent upgrades. https://status.prod.unique.app/incident/514786 Mon, 17 Feb 2025 15:34:00 -0000 https://status.prod.unique.app/incident/514786#a75645934aa4b6c744560ef6d1e81b8e9f0cb16b987b6433edeabc01fb5592c8 An unplanned security upgrade to a newer Kubernetes version during peak hours caused significant service degradations and disruptions for 1 hour and 33 minutes, affecting Unique.app's multitenant features. The incident was resolved by 17:47 CET after repairing document ingestion pipelines. To prevent future occurrences, we will implement stricter safeguards, allocate more resources for daytime operations, and segment workloads for independent upgrades. https://status.prod.unique.app/incident/514786 Mon, 17 Feb 2025 15:34:00 -0000 https://status.prod.unique.app/incident/514786#a75645934aa4b6c744560ef6d1e81b8e9f0cb16b987b6433edeabc01fb5592c8 An unplanned security upgrade to a newer Kubernetes version during peak hours caused significant service degradations and disruptions for 1 hour and 33 minutes, affecting Unique.app's multitenant features. The incident was resolved by 17:47 CET after repairing document ingestion pipelines. To prevent future occurrences, we will implement stricter safeguards, allocate more resources for daytime operations, and segment workloads for independent upgrades. https://status.prod.unique.app/incident/514786 Mon, 17 Feb 2025 15:34:00 -0000 https://status.prod.unique.app/incident/514786#a75645934aa4b6c744560ef6d1e81b8e9f0cb16b987b6433edeabc01fb5592c8 An unplanned security upgrade to a newer Kubernetes version during peak hours caused significant service degradations and disruptions for 1 hour and 33 minutes, affecting Unique.app's multitenant features. The incident was resolved by 17:47 CET after repairing document ingestion pipelines. To prevent future occurrences, we will implement stricter safeguards, allocate more resources for daytime operations, and segment workloads for independent upgrades. https://status.prod.unique.app/incident/514786 Mon, 17 Feb 2025 13:43:00 -0000 https://status.prod.unique.app/incident/514786#db2e2c7b22458a95e64d5a5b58346809bd3e75fed119de8e6c60243119dff9eb An unplanned security upgrade to a newer Kubernetes version during peak hours caused significant service degradations and disruptions for 1 hour and 33 minutes, affecting Unique.app's multitenant features. The incident was resolved by 17:47 CET after repairing document ingestion pipelines. To prevent future occurrences, we will implement stricter safeguards, allocate more resources for daytime operations, and segment workloads for independent upgrades. https://status.prod.unique.app/incident/514786 Mon, 17 Feb 2025 13:43:00 -0000 https://status.prod.unique.app/incident/514786#db2e2c7b22458a95e64d5a5b58346809bd3e75fed119de8e6c60243119dff9eb An unplanned security upgrade to a newer Kubernetes version during peak hours caused significant service degradations and disruptions for 1 hour and 33 minutes, affecting Unique.app's multitenant features. The incident was resolved by 17:47 CET after repairing document ingestion pipelines. To prevent future occurrences, we will implement stricter safeguards, allocate more resources for daytime operations, and segment workloads for independent upgrades. https://status.prod.unique.app/incident/514786 Mon, 17 Feb 2025 13:43:00 -0000 https://status.prod.unique.app/incident/514786#db2e2c7b22458a95e64d5a5b58346809bd3e75fed119de8e6c60243119dff9eb An unplanned security upgrade to a newer Kubernetes version during peak hours caused significant service degradations and disruptions for 1 hour and 33 minutes, affecting Unique.app's multitenant features. The incident was resolved by 17:47 CET after repairing document ingestion pipelines. To prevent future occurrences, we will implement stricter safeguards, allocate more resources for daytime operations, and segment workloads for independent upgrades. https://status.prod.unique.app/incident/514786 Mon, 17 Feb 2025 13:43:00 -0000 https://status.prod.unique.app/incident/514786#db2e2c7b22458a95e64d5a5b58346809bd3e75fed119de8e6c60243119dff9eb An unplanned security upgrade to a newer Kubernetes version during peak hours caused significant service degradations and disruptions for 1 hour and 33 minutes, affecting Unique.app's multitenant features. The incident was resolved by 17:47 CET after repairing document ingestion pipelines. To prevent future occurrences, we will implement stricter safeguards, allocate more resources for daytime operations, and segment workloads for independent upgrades. https://status.prod.unique.app/incident/514786 Mon, 17 Feb 2025 13:43:00 -0000 https://status.prod.unique.app/incident/514786#db2e2c7b22458a95e64d5a5b58346809bd3e75fed119de8e6c60243119dff9eb An unplanned security upgrade to a newer Kubernetes version during peak hours caused significant service degradations and disruptions for 1 hour and 33 minutes, affecting Unique.app's multitenant features. The incident was resolved by 17:47 CET after repairing document ingestion pipelines. To prevent future occurrences, we will implement stricter safeguards, allocate more resources for daytime operations, and segment workloads for independent upgrades. https://status.prod.unique.app/incident/514786 Mon, 17 Feb 2025 13:43:00 -0000 https://status.prod.unique.app/incident/514786#db2e2c7b22458a95e64d5a5b58346809bd3e75fed119de8e6c60243119dff9eb An unplanned security upgrade to a newer Kubernetes version during peak hours caused significant service degradations and disruptions for 1 hour and 33 minutes, affecting Unique.app's multitenant features. The incident was resolved by 17:47 CET after repairing document ingestion pipelines. To prevent future occurrences, we will implement stricter safeguards, allocate more resources for daytime operations, and segment workloads for independent upgrades. https://status.prod.unique.app/incident/514786 Mon, 17 Feb 2025 13:43:00 -0000 https://status.prod.unique.app/incident/514786#db2e2c7b22458a95e64d5a5b58346809bd3e75fed119de8e6c60243119dff9eb An unplanned security upgrade to a newer Kubernetes version during peak hours caused significant service degradations and disruptions for 1 hour and 33 minutes, affecting Unique.app's multitenant features. The incident was resolved by 17:47 CET after repairing document ingestion pipelines. To prevent future occurrences, we will implement stricter safeguards, allocate more resources for daytime operations, and segment workloads for independent upgrades. https://status.prod.unique.app/incident/514786 Mon, 17 Feb 2025 13:43:00 -0000 https://status.prod.unique.app/incident/514786#db2e2c7b22458a95e64d5a5b58346809bd3e75fed119de8e6c60243119dff9eb An unplanned security upgrade to a newer Kubernetes version during peak hours caused significant service degradations and disruptions for 1 hour and 33 minutes, affecting Unique.app's multitenant features. The incident was resolved by 17:47 CET after repairing document ingestion pipelines. To prevent future occurrences, we will implement stricter safeguards, allocate more resources for daytime operations, and segment workloads for independent upgrades. https://status.prod.unique.app/incident/514786 Mon, 17 Feb 2025 13:43:00 -0000 https://status.prod.unique.app/incident/514786#db2e2c7b22458a95e64d5a5b58346809bd3e75fed119de8e6c60243119dff9eb An unplanned security upgrade to a newer Kubernetes version during peak hours caused significant service degradations and disruptions for 1 hour and 33 minutes, affecting Unique.app's multitenant features. The incident was resolved by 17:47 CET after repairing document ingestion pipelines. To prevent future occurrences, we will implement stricter safeguards, allocate more resources for daytime operations, and segment workloads for independent upgrades. https://status.prod.unique.app/incident/514786 Mon, 17 Feb 2025 13:43:00 -0000 https://status.prod.unique.app/incident/514786#db2e2c7b22458a95e64d5a5b58346809bd3e75fed119de8e6c60243119dff9eb An unplanned security upgrade to a newer Kubernetes version during peak hours caused significant service degradations and disruptions for 1 hour and 33 minutes, affecting Unique.app's multitenant features. The incident was resolved by 17:47 CET after repairing document ingestion pipelines. To prevent future occurrences, we will implement stricter safeguards, allocate more resources for daytime operations, and segment workloads for independent upgrades. https://status.prod.unique.app/incident/514786 Mon, 17 Feb 2025 13:43:00 -0000 https://status.prod.unique.app/incident/514786#db2e2c7b22458a95e64d5a5b58346809bd3e75fed119de8e6c60243119dff9eb An unplanned security upgrade to a newer Kubernetes version during peak hours caused significant service degradations and disruptions for 1 hour and 33 minutes, affecting Unique.app's multitenant features. The incident was resolved by 17:47 CET after repairing document ingestion pipelines. To prevent future occurrences, we will implement stricter safeguards, allocate more resources for daytime operations, and segment workloads for independent upgrades. https://status.prod.unique.app/incident/514786 Mon, 17 Feb 2025 13:43:00 -0000 https://status.prod.unique.app/incident/514786#db2e2c7b22458a95e64d5a5b58346809bd3e75fed119de8e6c60243119dff9eb An unplanned security upgrade to a newer Kubernetes version during peak hours caused significant service degradations and disruptions for 1 hour and 33 minutes, affecting Unique.app's multitenant features. The incident was resolved by 17:47 CET after repairing document ingestion pipelines. To prevent future occurrences, we will implement stricter safeguards, allocate more resources for daytime operations, and segment workloads for independent upgrades.