11-22-2022
Outcomes
Summary
Artifacts
- Explaining what happened in incident-385 202211220922 for friends:
- Ubuntu kernel bug
- The service we use for managing containers on Linux, runc, uses a syscall called epoll for managing sysout.
- Waker, which is the software in the kernel that’s responsible for waking up the runc thread, had a bug, which means that runc doesn’t know that the stdout pipe is full, which means that it doesn’t attempt to drain it.
- This causes workers to block on trying to write to the pipe.
Notes
- Figure out where the test coverage in enterprise_membership.tsx is.
- Upcoming PRs:
uid: 202211220801 tags: #journal