-
Notifications
You must be signed in to change notification settings - Fork 2k
fix(subscriptions): cap png export height to prevent OOM #40754
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Conversation
| """ | ||
| ) | ||
|
|
||
| if max_height_pixels and height > max_height_pixels: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The caveat with this approach is that chrome still has to load the entire table, but at least it doesn't have to screenshot the entire table.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Additional Comments (1)
-
ee/tasks/subscriptions/subscription_utils.py, line 122 (link)logic: Celery task path doesn't pass height limit, so
generate_assets()(used by some subscriptions and Slack sharing) won't get OOM protection thatgenerate_assets_async()has
3 files reviewed, 2 comments
Problem
There are two issues with exporting images:
There are some large insights (tables with thousands of rows or giant rows) that eat up memory when we try to screenshot them.
In addition, the thread running the screenshot sometimes continues even after the top level coroutine times out, resulting in 100% memory usage after the temporal workflow has completed. Temporal is able to retry activities when we the pods get OOM killed, but if there are too many OOMs in a single workflow, the retries get exhausted.
related convo: https://posthog.slack.com/archives/C09BDQLM8KA/p1761925099100499
Changes
Address, the first issue directly by limiting the number of pixels we screenshot. This should indirectly help the second issue, as the screenshot shouldn't run as long with fewer pixels to capture.
How did you test this code?
👉 Stay up-to-date with PostHog coding conventions for a smoother review.
Locally. Verified that a png export is shorter with the new setting.
Changelog: (features only) Is this feature complete?