Skip to content

Add memory consumption tracking to df execution report #1668

@norberttech

Description

@norberttech

Right now when analyze option is configured and passed to df()->run() dataframe following statistics are goign to be collected:

  • execution time (hrtime)
  • total processed rows (int)
  • schema (optionally collected, when configured)
  • column statistics (optionally collected when configured)

Another interesting metric that I believe we can collect next to hrtime is memory consumption through memory_get_usage(true)

That will simplify monitoring of pipelines

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    Projects

    Status

    Done

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions