Skip to content

Performance Bug in the varmap function #1592

Closed
@kohlmanj-hs

Description

@kohlmanj-hs

Describe the bug: ...
When python code throws an error, and one of the local variables contains a very large list of lists of lists, it takes the elastic python apm client a very long time to process through the list. In our case it was taking over 60 seconds for the code to complete.

To Reproduce
To simply replicate the issue, I: (see code snipit in the elastic forum, link below)

  • created a simple script to create a list of list of lists of integers.
  • in one method I had my "with elasticAPMTransaction:" statement.
  • under this I created the list (200 integers) of list (300 integers) of list (40 integers) aka 24M items total.
  • started to iterate through the outer list and picked an arbitrary spot to through an exception. This essentially mimicked the method where the real code threw an exception.
  • wrapped this whole thing in a profiler and evaluated the output.
    This would take 60 seconds +/- to complete.

Seems like the issue is in the varmap function in elasticapm\utils_init_.py but there maybe lots of ways to solve this issue.

Environment (please complete the following information)

  • OS: [e.g. Linux]
  • Python version: 3.x
  • Framework and version [e.g. Django 2.1]:
  • APM Server version: N/A
  • Agent version: 6.7.2

Additional context
Created a case here https://discuss.elastic.co/t/python-client-apm-very-slow-when-local-list-variable-is-very-large/309935. See this for additional details.

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions